Elamur Research⌕
☰
home / papers / 2026 / Does Translation Preserve Sentiment? An Analysis of Arabic-English Cross-Lingual Classification

Does Translation Preserve Sentiment? An Analysis of Arabic-English Cross-Lingual Classification

Paper · 2026 · Nour Aldin Al Mubarak and Noura Al Moubayed

Summary

Many cross-lingual sentiment systems translate text first and then classify it, assuming that translation keeps the original polarity. This paper tests that assumption for Arabic and English. The authors used 11,558 labelled samples from three corpora: AJGT (Jordanian tweets) and OCLAR (Lebanese reviews), translated Arabic to English, and FSA (financial sentences), translated English to Arabic. Each text was translated by Helsinki-NMT, GPT-4o-mini and LLaMA-3.1-8B. One multilingual classifier, XLM-RoBERTa, then scored both the source and the translated versions, so that any change in prediction can be attributed to translation rather than to switching classifiers.

Sentiment changes after translation were common. Accuracy losses ranged from under 1% to close to 20%, depending on the dataset and the translator. GPT-4o-mini preserved sentiment best. LLaMA-3.1-8B both distorted sentiment and sometimes refused to translate. Because Helsinki-NMT translated every input, these refusals appear to come from safety tuning rather than from untranslatable content. The measured shifts also changed when a different evaluation classifier was used. Together, these results caution against treating translate-then-classify as a neutral step in Arabic NLP pipelines.

Key points

  • Data: 11,558 samples in total.
    • AJGT: 1,800 tweets, balanced positive/negative, translated AR→EN.
    • OCLAR: 3,916 reviews on a 3-class scale derived from star ratings, translated AR→EN.
    • FSA (Malo et al. financial phrase bank): 5,842 sentences, 3 classes, translated EN→AR.
  • Translators: Helsinki-NMT (opus-mt-ar-en / opus-mt-en-ar), GPT-4o-mini via API, and LLaMA-3.1-8B run locally. Both LLMs used a single fixed prompt at temperature 0.
  • Classifier and metric: cardiffnlp/twitter-xlm-roberta-base-sentiment was the fixed classifier. Its prediction on the original text serves as the baseline. The Sentiment Shift Rate (SSR) is the share of samples whose prediction changes after translation.
  • Label handling for AJGT: neutral predictions were reassigned to whichever polar class (positive or negative) had the higher confidence.
  • Results: accuracy drops spanned less than 1% to almost 20%. GPT-4o-mini was the most faithful translator. LLaMA-3.1-8B showed safety-driven refusals, which no Helsinki-NMT output exhibited. Shift estimates depended on which classifier was used for evaluation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Durham UniversityDurham · United Kingdom8 works · 561 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 4Research subjects and languages the work is about.English · 2,263 worksSentiment & emotion · 972 worksSemitic languages · 2,908 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Nour Aldin Al Mubarak and Noura Al Moubayed. 2026. Does Translation Preserve Sentiment? An Analysis of Arabic-English Cross-Lingual Classification. The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2khexrk5s3bx

@inproceedings{mubarak2026does,
  title = {{Does Translation Preserve Sentiment? An Analysis of Arabic-English Cross-Lingual Classification}},
  author = {Nour Aldin {Al Mubarak} and Noura {Al Moubayed}},
  booktitle = {The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks},
  year = {2026},
  doi = {10.63317/2khexrk5s3bx},
  url = {https://doi.org/10.63317/2khexrk5s3bx}
}

References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, Mona Diab, and Sandra Kübler. 2014. SAMAR: Subjectivity and sentiment analysis for Arabic social media. Computer Speech & Language, 28(1):20–37. ↗
  2. Marwan Al Omari, Moustafa Al-Hajj, Nacereddine Hammami, and Amani Sabra. 2019. Sentiment classifier: Logistic regression for Arabic services’ reviews in Lebanon. 2019 International Conference on Computer and Information Sciences, ICCIS 2019. find
  3. Khaled Mohammad Alomari, Hatem M. ElSherif, and Khaled Shaalan. 2017. Arabic tweets sentimental analysis using machine learning. In Proceedings of IEA/AIE 2017, pages 602–610. Springer. find
  4. Alexandra Balahur and Marco Turchi. 2014. Comparative experiments using supervised learning and machine translation for multilingual sentiment analysis. Computer Speech & Language, 28(1):56–75. ↗
  5. Francesco Barbieri, Luis Espinosa Anke, and Jose Camacho-Collados. 2022. XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond.
  6. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2024. Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions. 12th International Conference on Learning Representations, ICLR 2024. ↗
  7. Alessio Buscemi and Daniele Proverbio. 2024. ChatGPT vs Gemini vs LLaMA on Multilingual Sentiment Analysis. ↗
  8. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 8440–8451. find
  9. Jingfeng Cui, Zhaoxia Wang, Seng Beng Ho, and Erik Cambria. 2023. Survey on sentiment analysis: evolution of research methods and topics. Artificial Intelligence Review, 56(8):8469–8510. ↗
  10. Jean Marc Dewaele, K. V. Petrides, and Adrian Furnham. 2008. Effects of trait emotional intelligence and sociobiographical variables on communicative anxiety and foreign language anxiety among adult multilinguals: A review and empirical investigation. Language Learning, 58(4):911– 960. ↗
  11. Geetika Gautam and Divakar Yadav. 2014. Sentiment analysis of twitter data using machine learning approaches and semantic analysis. In 2014 7th International Conference on Contemporary Computing, IC3 2014, pages 437–442. Institute of Electrical and Electronics Engineers Inc. find
  12. Md Arid Hasan. 2024. Ensemble Language Models for Multilingual Sentiment Analysis. ↗
  13. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalisation. find
  14. Phillip Keung, Yichao Lu, György Szarvas, and Noah A. Smith. 2020. The Multilingual Amazon Reviews Corpus. EMNLP 2020 - 2020 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pages 4563–4568.
  15. Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ali Payani, Ninghao Liu, and Mengnan Du. 2024. Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages.
  16. AI @ Meta Llama Team. 2024. The Llama 3 Herd of Models.
  17. P Lohar, H Afli, A Way Proceedings of the 13th Conference of the, and undefined 2018. 2018. Balancing translation quality and sentiment preservation (Non-archival Extended Abstract). aclanthology.orgP Lohar, H Afli, A WayProceedings of the 13th Conference of the Association for Machine, 2018•aclanthology.org. find
  18. Pekka Malo, Ankur Sinha, Pekka Korhonen, Jyrki Wallenius, and Pyry Takala. 2014. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65(4):782– 796. ↗
  19. Md Saef Ullah Miah, Md Mohsin Kabir, Talha Bin Sarwar, Mejdl Safran, Sultan Alfarhood, and M F Mridha. 2024. A multimodal approach to cross-lingual sentiment analysis with ensemble of transformer and LLM. Scientific Reports, 14(1). ↗
  20. Saif M. Mohammad, Mohammad Salameh, and Svetlana Kiritchenko. 2016. How translation alters sentiment. Journal of Artificial Intelligence Research, 55:95–130. ↗
  21. OpenAI Team OpenAI. 2024. Gpt-4 technical report. ↗
  22. Oumaima Oueslati, Erik Cambria, Moez Ben HajHmida, and Habib Ounelli. 2020. A review of sentiment analysis research in Arabic language. Future Generation Computer Systems, 112:408– 430.
  23. Maja Popović. 2015. chrF: character n-gram Fscore for automatic MT evaluation. 10th Workshop on Statistical Machine Translation, WMT 2015 at the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015 - Proceedings, pages 392–395. find
  24. Hadeel Saadany, Constantin Orăsan, Emad Mohamed, and Ashraf Tantavy. 2021. Sentiment-aware measure (sam) for evaluating sentiment transfer by machine translation systems. In Proceedings of the Conference Recent Advances in Natural Language Processing - Deep Learning for Natural Language Processing Methods and Applications, pages 1217–1226. INCOMA Ltd. Shoumen, BULGARIA. find
  25. Jörg Tiedemann and Santhosh Thottingal. 2020. OPUS-MT – Building open translation services for the World. find
  26. Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao, and Michael R. Lyu. 2024. All Languages Matter: On the Multilingual Safety of Large Language Models. ↗
  27. Mohamed Zouidine and Mohammed Khalil. 2025. Large Language Models for Arabic Sentiment Analysis and Machine Translation. Engineering, Technology & Applied Science Research, 15(2):20737–20742. 10. ↗
  28. Language Resource References Al Omari, M., Al-Hajj, M., Hammami, N., and Sabra, A. (2019). Opinion Corpus for Lebanese Arabic Reviews (OCLAR). Sentiment analysis corpus of 3,916 Lebanese Arabic product and service reviews with 3-class labels (positive, neutral, negative). UCI Machine Learning Repository. https://archive.ics.uci. edu/dataset/499/opinion+corpus+for+ lebanese+arabic+reviews+oclar Alomari, K. M., ElSherif, H. M., and Shaalan, K. (2017). Arabic Jordanian General Tweets (AJGT) Dataset. Sentiment corpus of 1,800 Jordanian Arabic tweets. https://github.com/komari6/Arabic-twitter-corpus-AJGT ↗
  29. Barbieri, F., Espinosa Anke, L., and Camacho-Collados, J. (2022). XLM-RoBERTa Sentiment Model (twitter-xlm-roberta-base-sentiment). Multilingual sentiment classifier, pre-trained on 198M tweets. https://huggingface.co/cardiffnlp/twitter-xlm-roberta-base-sentiment ↗
  30. Malo, P., Sinha, A., Korhonen, P., Wallenius, J., and Takala, P. (2014). Financial Sentiment Analysis (FSA) Dataset. Corpus of 5,842 English financial news sentences with 3-class labels. Published in: Malo et al. (2014), JASIST 65(4):782–796. https://doi.org/10.1002/asi.23062 ↗
  31. Tiedemann, J. and Thottingal, S. (2020). OPUS-MT Arabic-English Models (opus-mt-ar-en, opus-mt-en-ar). Bidirectional Arabic-English neural translation models. https://huggingface.co/Helsinki-NLP/opus-mt-ar-en ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mubarak-2026-does-translation-preserve-sentiment-analysis/