Elamur Research⌕
☰
home / papers / 2026 / L3IA at AraSentEval 2026 Subtask 2: LLM-Based Multi-Step Pipeline for Arabic Sentiment Swap

L3IA at AraSentEval 2026 Subtask 2: LLM-Based Multi-Step Pipeline for Arabic Sentiment Swap

Paper · 2026 · Abdessamad Benlahbib, Hamza Alami, Mohamed M'haouach et al.

Summary

This system paper targets Subtask 2 of AraSentEval 2026: given an Arabic sentence in MSA or a dialect, rewrite it so its polarity flips (positive ↔ negative) while the rest of the content stays intact. The authors did no fine-tuning. Instead, they chain three prompted calls to an OpenAI model:

  1. Locate every sentiment-carrying unit: adjectives, emotive verbs, intensifiers, idioms and negations.
  2. Propose an opposite for each unit, respecting agreement with its context.
  3. Reassemble the sentence, keeping the original word order and making as few changes as possible.

Each step returns JSON checked against Pydantic schemas, and an asyncio layer runs samples in parallel. On the official test set, 74.3% of the system's outputs are judged to carry the target sentiment, and it scores 27.22 on BLEU and 55.04 on chrF.

The authors argue the staged design makes the process transparent, since each intermediate result can be inspected. It also needs no training data or GPUs. Its weaknesses are error cascades from missed expressions, insertion of extra words, poor emoji handling, and the cost of three API calls per sample.

Key points

  • The data comes from MA'AKS, a hand-curated parallel sentiment-swap corpus. It covers MSA plus Egyptian, Gulf, Levantine and Maghrebi dialects, drawn from social-media text.
  • The model is OpenAI GPT-5-Nano, called through LangChain. Outputs are typed objects (SentimentAnalysis, OppositeExpression, SwappedSentence) validated with Pydantic.
  • Official evaluation has two parts:
    • Polarity is judged by the classifier CAMeL-Lab/bert-base-arabic-camelbert-da-sentiment.
    • Content retention is measured by BLEU and chrF against the source sentence.
  • Test results are 74.3% accuracy, 27.22 BLEU and 55.04 chrF. No few-shot examples or training data were used.
  • Reported failure modes:
    • Missed expressions in step 1 carry through to later steps.
    • The reconstruction step sometimes adds new words.
    • Emojis are turned into text descriptions instead of being flipped.
    • Each sample costs three API calls.

Interesting ideas

  • I1. Splitting polarity inversion into locate → substitute-with-agreement → minimally reassemble keeps edits local, so neutral content survives untouched.
  • I2. The extraction step treats negators and intensifiers as sentiment units in their own right, next to adjectives and verbs. Each unit is tagged with its position, direction and an importance level.
  • I3. Every intermediate step must emit schema-validated structured output, so failures can be traced to a single stage.
  • I4. The substitution step reports for itself whether the replacement fits in place, and notes any adaptation it needed.
  • I5. The paper points out that source-overlap metrics such as BLEU and chrF reward making too few changes, so they must be read alongside a polarity check.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): when regenerating text from a meaning graph with one evaluative attribute flipped, Mova could replace only the affected nodes and reuse the original surface for everything else.
  • I2 (frames): the prag crate could attach evaluative-polarity and negation-scope marks to UD subtrees with an importance weight, which would feed irony detection and implicature.
  • I3 (learn): LLM teacher tiers could be required to return typed, validated records, with malformed answers rejected before they reach silver data.
  • I4 (back): Mova's generator could emit a fit/agreement flag per substituted lemma, sending flagged slots to morphology re-inflection instead of trusting the raw swap.
  • I5 (learn): Mova's evaluation could pair every preservation metric with a "copy the input" negative control, so a score that rewards doing nothing becomes visible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Sidi Mohamed Ben Abdellah UniversityFes · Morocco8 works · 156 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Abdessamad Benlahbib, Hamza Alami, Mohamed M'haouach and Kaouthar Elyoussoufi. 2026. L3IA at AraSentEval 2026 Subtask 2: LLM-Based Multi-Step Pipeline for Arabic Sentiment Swap. The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4wtc4onqmfgo

@inproceedings{benlahbib2026l3ia,
  title = {{L3IA at AraSentEval 2026 Subtask 2: LLM-Based Multi-Step Pipeline for Arabic Sentiment Swap}},
  author = {Abdessamad Benlahbib, Hamza Alami, Mohamed M'haouach and Kaouthar Elyoussoufi},
  booktitle = {The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks},
  year = {2026},
  doi = {10.63317/4wtc4onqmfgo},
  url = {https://doi.org/10.63317/4wtc4onqmfgo}
}

References

13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saad Ezzini, Shadi Abudalfa, Maram Alharbi, Salmane Chafik, Hamzah Luqman, Mo El-Haj, Paul Rayson, Samaher Alghamdi, and Reem Alotaibi. 2026. AraSentEval: A shared task on sentiment analysis and swapping in arabic. In Proceedings of the 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7), co-located with the Language Resources and Evaluation Conference (LREC 2026), Palma, Mallorca, Spain. find
  2. Soufiane Hajbi, Younes Chihab, Rachid Ed-Dali, and Redouan Korchiyne. 2022. Natural language processing based approach to overcome arabizi and code switching in social media moroccan dialect. In Advances in Information, Communication and Cybersecurity, pages 57– 66, Cham. Springer International Publishing. find
  3. Zhiqiang Hu, Roy Ka-Wei Lee, Charu C. Aggarwal, and Aston Zhang. 2022. Text style transfer: A review and experimental evaluation. SIGKDD Explor. Newsl., 24(1):14–45. find
  4. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Online). Association for Computational Linguistics.
  5. Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text Output Notes Negation removal + positive restructuring ةﺪﻛ ﺪﻌﺑو ﻞﻳﺎﺑﻮﳌا ﻢﻗر ﺖﻠﺧدو ﻦﺸﻴﻜﻠﺑﻻا ﺖﻟﺰﻧ ﺎﻧا ! حﺎﺠﻨﺑ ﰎ ﻲﻟﺎﻗ ﺔﻈﻓﺎﶈا ﻞﺧدا ﺖﻴﺟ Minimal change, good preservation style transfer: A survey. Computational Linguistics, 48(1):155–205. find
  6. Aditya Joshi, Raj Dabre, Diptesh Kanojia, Zhuang Li, Haolan Zhan, Gholamreza Haffari, and Doris Dippold. 2025. Natural language processing for dialects of a language: A survey. ACM Comput. Surv., 57(6). ↗
  7. Raed Mughaus, Shadi Abudalfa, Hamzah Luqman, Fahad Abdu, Mohammed AlAli, Nawaf Al-Dowayan, and Ahmed Abdelali. 2026. Ma’aks: manually-curated parallel dataset for arabic text sentiment swap. Language Resources and Evaluation, 60(1):1. ↗
  8. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. ↗
  9. Maja Popović. 2015. chrF: character n-gram Fscore for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. find
  10. Martina Toshevska and Sonja Gievska. 2022. A review of text style transfer using deep learning. IEEE Transactions on Artificial Intelligence, 3(5):669–684. ↗
  11. Martina Toshevska and Sonja Gievska. 2025. Llm-based text style transfer: Have we taken a step forward? IEEE Access, 13:44707–44721. ↗
  12. Jihene Younes, Emna Souissi, Hadhemi Achour, and Ahmed Ferchichi. 2020. Language resources for maghrebi arabic dialects’nlp: a survey. Language Resources and Evaluation, 54(4):1079–1142. find
  13. Tianshan Zhang and Hao Tang. 2025. Style transfer: A decade survey. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/benlahbib-2026-l3ia-arasenteval-2026-subtask-2/