Elamur Research⌕
☰
home / papers / 2025 / Dial2MSA-Verified: A Multi-Dialect Arabic Social Media Dataset for Neural Machine Translation to Modern Standard Arabic

Dial2MSA-Verified: A Multi-Dialect Arabic Social Media Dataset for Neural Machine Translation to Modern Standard Arabic

Paper · 2025 · Abdullah Khered, Youcef Benkhedda and Riza Batista-Navarro

Summary

Arabic social media text mixes Modern Standard Arabic (MSA) with regional dialects, nonstandard spelling, slang and foreign words. Normalising it into MSA is hard because parallel DA–MSA data is scarce. The only public multi-dialect social media resource, Dial2MSA, had verified translations for just two of its four dialects. The authors close that gap by verifying the Gulf (GLF) and Levantine (LEV) portions. They first filter and auto-correct candidate translations, then have three native-speaker annotators per dialect make the final check. The result, Dial2MSA-Verified, has fixed train, validation and test splits, and each tweet carries several MSA references. The authors then fine-tune Seq2Seq NMT models on it, adding parallel data from other domains. Scoring against multiple references gives higher results than scoring against a single one. Training jointly across dialects works better than per-dialect models chosen through dialect identification. AraT5 performs best overall. The resource gives researchers a verified benchmark for normalising dialectal Arabic on social media.

Key points

  • The original Dial2MSA sampled 24,000 tweets (6,000 per dialect: EGY, MGR, LEV, GLF) from 175 million collected tweets. Only EGY and MGR had been verified there.
  • Six native Arabic speakers verified the LEV and GLF translations, three per dialect. The final split is 18,991 training, 800 validation and 8,000 test tweets, each with multiple MSA references.
  • Before human review, candidate pairs went through several steps:
    • removing duplicates;
    • dropping pairs whose MSA side contained non-Arabic or known dialectal words;
    • dropping pairs with very different segment counts;
    • automatic correction with an Arabic spelling tool and GPT-4.
  • The J-R configuration (joint training) beat I-R. I-R depended on dialect identification to pick a model, and identification errors hurt translation. AraT5 averaged BLEU 41.12 and chrF++ 62.05. AraT5v2 did best on GLF.
  • In error analysis, the model handled Arabised English social media terms well (followers, Facebook, retweet). It struggled with culture-bound idioms such as a LEV phrase meaning "lucky you", with Algerian-specific words, and with exam names in Arabic script. Sudanese and Yemeni dialects are not covered.

Interesting ideas

  • I1. Scoring a system against several valid references, rather than one, avoids penalising correct but differently worded outputs and gave a fairer picture of model quality.
  • I2. Cheap mechanical filters run before any human sees the data: deduplication, a blocklist of target-side words that signal a bad translation, and a length/segment-ratio check.
  • I3. Automatic correction comes first (a spelling tool, then GPT-4), and humans only confirm or reject the result, which keeps annotators as final judges and cuts their workload.
  • I4. One model trained jointly on related varieties outperformed separate per-variety models chosen by an identifier, because the varieties share a lot and routing mistakes cascade.
  • I5. The error analysis separates what transferred well (borrowed technical terms) from what failed (idioms with culture-specific connotation, words that look native but are foreign).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation of generated paraphrases or answers could accept a set of gold variants, so that a correct alternative wording is not counted as a failure.
  • I2 (learn): Silver data from witness parsers could pass similar sanity gates, such as token-count mismatches or forbidden labels, before reaching the agreement vote or an LLM teacher.
  • I3 (learn): This fits the LLM teacher tiers: let the LLM propose repairs to silver trees and keep human or stricter checks only as the last verification layer.
  • I4 (tree): Rather than routing text to genre- or dialect-specific parsers, gparse could be trained on pooled English treebanks (standard plus web/social varieties), avoiding a fragile classifier step in front.
  • I5 (frames): Idiomatic expressions like "lucky you" need dedicated pragmatic handling rather than compositional role assignment, so prag could keep an explicit idiom/speech-act lexicon for such cases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 works

Cite

Cite as Abdullah Khered, Youcef Benkhedda and Riza Batista-Navarro. 2025. Dial2MSA-Verified: A Multi-Dialect Arabic Social Media Dataset for Neural Machine Translation to Modern Standard Arabic. Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{khered2025dial2msa,
  title = {{Dial2MSA-Verified: A Multi-Dialect Arabic Social Media Dataset for Neural Machine Translation to Modern Standard Arabic}},
  author = {Abdullah Khered, Youcef Benkhedda and Riza Batista-Navarro},
  booktitle = {Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4)},
  year = {2025},
  url = {https://aclanthology.org/2025.wacl-1.6/}
}

References

50 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Laith H. Baniata, Isaac. K. E. Ampomah, and Seyoung Park. 2021. A transformer-based neural machine translation model for arabic dialects that utilizes subword units. Sensors, 21(19). find
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  3. Nouhaila Bensalah, Habib Ayad, Abdellah Adib, and Abdelhamid Ibn El Farouk. 2021. Transformer model and convolutional neural networks (cnns) for arabic to english machine translation. In International Conference On Big Data and Internet of Things, pages 399–410. Springer. find
  4. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, El Moatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. Nadi 2023: The fourth nuanced arabic dialect identification shared task. Preprint, arXiv:2310.16117. ↗
  5. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The madar arabic dialect corpus and lexicon. In International Conference on Language Resources and Evaluation. find
  6. Muhammad Abdul-Mageed, Amr Keleg, AbdelRahim Elmadany, Chiyu Zhang, Injy Hamed, Walid Magdy, Houda Bouamor, and Nizar Habash. 2024. Nadi 2024: The fifth nuanced arabic dialect identification shared task. Preprint, arXiv:2407.04910. ↗
  7. Mohammed ElAmine Chennafi, Hanane Bedlaoui, Abdelghani Dahou, and Mohammed A. A. Al-qaness. 2022. Arabic aspect-based sentiment classification using seq2seq dialect normalization and transformers. Knowledge, 2(3):388–401. ↗
  8. G. H. Al-Gaphari and M. Al-Yadoumi. 2010. A method to convert sana’ani accent to modern standard arabic. find
  9. Rehab M. Duwairi. 2015. Sentiment analysis for dialectical arabic. In 2015 6th International Conference on Information and Communication Systems (ICICS), pages 166–170. find
  10. Mohamed Osman Hegazi, Yasser Al-Dossari, Abdullah Al-Yahy, Abdulaziz Al-Sumari, and Anwer Hilal. 2021. Preprocessing arabic text on social media. Heliyon, 7(2). ↗
  11. Moussa Kamal Eddine, Nadi Tomeh, Nizar Habash, Joseph Le Roux, and Michalis Vazirgiannis. 2022. Arabart: a pretrained arabic sequence-to-sequence model for abstractive summarization. arXiv preprint arXiv:2203.10945. ↗
  12. Abdullah Khered, Ingy Abdelhalim, Nadine Abdelhalim, Ahmed Soliman, and Riza Theresa Batista-Navarro. 2023. Unimanc at nadi 2023 shared task: A comparison of various t5-based models for translating arabic dialectical text to modern standard arabic. In Proceedings of ArabicNLP 2023, pages 658–664.
  13. AbdelRahim Elmadany, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. Octopus: A multitask model and toolkit for arabic natural language generation. Preprint, arXiv:2310.16127. ↗
  14. Abdullah Salem Khered, Ingy Yasser Hassan Abdou Abdelhalim, and Riza Batista-Navarro. 2022. Building an ensemble of transformer models for Arabic dialect classification and sentiment analysis. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 479–484, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  15. Ashraf Hatim Elneima, AhmedElmogtaba Abdelmoniem Ali Abdelaziz, and Kareem Darwish. 2024. Osact6 dialect to msa translation shared task overview. In Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation@ LREC-COLING 2024, pages 93–97. find
  16. M Lewis. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. ↗
  17. GÜLŞEN ERYİǦİT and DİLARA TORUNOǦLU-SELAMET. 2017. Social media text normalization for turkish. Natural Language Engineering, 23(6):835–875. ↗
  18. Y Liu. 2020. Multilingual denoising pre-training for neural machine translation. arXiv preprint arXiv:2001.08210. ↗
  19. Fatma Mallek, Billal Belainine, and Fatiha Sadat. 2017. Arabic social media analysis and translation. Procedia Computer Science, 117:298–303. Arabic Computational Linguistics. find
  20. Mohamed Atta Faheem, Khaled Tawfik Wassif, Hanaa Bayomi, and Sherif Mahdy Abdou. 2024. Improving neural machine translation for low resource languages through non-parallel corpora: a case study of egyptian dialect to modern standard arabic translation. Scientific Reports, 14(1):2265. ↗
  21. Claudia Matos Veliz, Orphée De Clercq, and Veronique Hoste. 2021. Is neural always better? smt versus nmt for dutch text normalization. Expert Systems with Applications, 170:114500. ↗
  22. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. find
  23. Karima Meftouh, Salima Harrat, and Kamel Smaïli. 2018. PADIC: extension and new experiments. In 7th International Conference on Advanced Technologies ICAT. find
  24. Mahmoud Ghoneim and Mona Diab. 2013. Multiword expressions in the context of statistical machine translation. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 1181–1187. ↗
  25. Hamdy Mubarak. 2018. Dial2msa: A tweets corpus for converting dialectal arabic to modern standard arabic. OSACT, 3:49. find
  26. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, and Teven et al. Le Scao. 2023. Crosslingual generalization through multitask finetuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15991– 16111, Toronto, Canada. Association for Computational Linguistics. ↗
  27. Salwa Hamada and Reham Marzouk. 2018. Developing a Transfer-Based System for Arabic Dialects Translation, pages 121–138. find
  28. Ahmed Hamdi, Rahma Boujelbane, Nizar Habash, and Alexis Nasr. 2013. The effects of factorizing root and pattern mapping in bidirectional tunisian-standard arabic machine translation. In Proceedings of Machine Translation Summit XIV: Papers.
  29. El Moatez Billah Nagoudi, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2022. AraT5: Text-to-text transformers for Arabic language generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 628–647, Dublin, Ireland. Association for Computational Linguistics.
  30. Salima Harrat, Karima Meftouh, and Kamel Smaili. 2019. Machine translation for arabic dialects (survey). Information Processing Management, 56(2):262–273. Advance Arabic Natural Language Processing (ANLP) and its Applications. find
  31. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 348–358.
  32. Wael Salloum and Nizar Habash. 2014. Adam: Analyzer for dialectal arabic morphology. Journal of King Saud University-Computer and Information Sciences, 26(4):372–378. ↗
  33. Maja Popović. 2017. chrF++: words helping character n-grams. In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics. ↗
  34. Hassan Sawaf. 2010. Arabic dialect handling in hybrid machine translation. In Proceedings of the 9th Conference of the Association for Machine Translation in the Americas: Research Papers, Denver, Colorado, USA. Association for Machine Translation in the Americas.
  35. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Belgium, Brussels. Association for Computational Linguistics. ↗
  36. Pamela Shapiro and Kevin Duh. 2019. Comparing pipelined and integrated approaches to dialectal Arabic neural machine translation. In Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects, pages 214–222, Ann Arbor, Michigan. Association for Computational Linguistics.
  37. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. ↗
  38. Amel Slim and Ahlem Melouah. 2024. Low resource arabic dialects transformer neural machine translation improvement through incremental transfer of shared linguistic features. Arabian Journal for Science and Engineering, pages 1–17. ↗
  39. Sandra Rizkallah, Amir Atiya, Hossam ElDin Mahgoub, and Momen Heragy. 2018. Dialect versus msa sentiment analysis. In The International Conference on Advanced Machine Learning Technologies and Applications (AMLTA2018), pages 605–613. Springer. find
  40. Amel Slim, Ahlem Melouah, Usef Faghihi, and Khouloud Sahib. 2022. Improving neural machine translation for low resource algerian dialect by transductive transfer learning strategy. Arabian Journal for Science and Engineering, 47(8):10411–10418. ↗
  41. Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. 2014. Automatic identification of arabic dialects in social media. SoMeRA ’14, page 35–40, New York, NY, USA. Association for Computing Machinery. find
  42. Ridouane Tachicart and Karim Bouzoubaa. 2014. A hybrid approach to translate moroccan arabic dialect. In 2014 9th International Conference on Intelligent Systems: Theories and Applications (SITA-14), pages 1–5. find
  43. Hassan Sajjad, Kareem Darwish, and Yonatan Belinkov. 2013. Translating dialectal arabic to english. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1–6. find
  44. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational Linguistics. ↗
  45. Wael Salloum, Heba Elfardy, Linda Alamir-Salloum, Nizar Habash, and Mona Diab. 2014. Sentence level dialect identification for machine translation system selection. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 772–778. find
  46. Wael Salloum and Nizar Habash. 2011. Dialectal to standard arabic paraphrasing to improve arabic-english statistical machine translation. In Proceedings of the first workshop on algorithms and resources for modelling of dialects and language varieties, pages 10–21. ↗
  47. Jezia Zakraoui, Moutaz Saleh, Somaya Al-Maadeed, and Jihad Mohamed Alja’am. 2021. Arabic machine translation: A survey with challenges and future directions. IEEE Access, 9:161445–161468. ↗
  48. Randa Zarnoufi, Hamid Jaafar, and Mounia Abik. 2020. Machine normalization: Bringing social media text from non-standard to standard form. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 19(4). find
  49. Wael Salloum and Nizar Habash. 2012. Elissa: A dialectal to standard arabic machine translation system. In International Conference on Computational Linguistics.
  50. Wael Salloum and Nizar Habash. 2013. Dialectal arabic to english machine translation: Pivoting through modern standard arabic. In Proceedings of the 2013

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khered-2025-dial2msa-verified-multi-dialect-arabic/