Elamur Research⌕
☰
home / papers / 2025 / A Multi-Task Learning Approach to Dialectal Arabic Identification and Translation to Modern Standard Arabic

A Multi-Task Learning Approach to Dialectal Arabic Identification and Translation to Modern Standard Arabic

Paper · 2025 · Abdullah Khered, Youcef Benkhedda and Riza Batista-Navarro

Summary

The paper addresses the task of translating Arabic dialects (DA) into the standard written form of the language (MSA). This is hard because dialects differ in morphology and vocabulary, and social media text adds slang, code-switching and informal spelling. The authors attach a dialect-identification objective to a translation model. A pre-trained Arabic encoder-decoder (AraT5 or AraBART) shares its encoder between an MSA-generating decoder and a small classification head. The head predicts one of five labels: EGY, GLF, LEV, MGR or MSA. Both cross-entropy losses are combined with weights into a single training objective. They also build LahjaTube, a corpus made from Creative Commons YouTube videos in the four dialect regions. Its English subtitles were rendered into MSA by GPT-4o. They compare single-task (STL) and multi-task (MTL) setups on the multi-reference Dial2MSA-Verified tweet benchmark. The multi-task setup, especially with LahjaTube added to training, raises BLEU by 2.65 points over the baselines. The work shows that cheap auxiliary labels and noisy, automatically built parallel data can help a low-resource translation task.

Key points

  • The architecture has a shared Transformer encoder feeding two branches: a pre-trained decoder for MSA output and a linear-plus-softmax head for dialect classification.
  • The best translation model is MTL-AraT5 trained on the largest data mix (Training Subset 3), with BLEU 43.77 and chrF++ 63.96 on Dial2MSA-Verified.
  • The best dialect classifier is MTL-AraBART (Training Subset 2), with 98.85% accuracy and 99.10% weighted-F1. The authors note that translation training appears to help classification as well.
  • LahjaTube covers EGY, GLF, LEV and MGR and includes DA transcripts with English and MSA versions:
    • MSA was produced by GPT-4o from the English side.
    • Only 200 DA-MSA pairs were checked by humans.
  • A notable error mode: when the model wrongly labels a dialectal input as MSA, it tends to copy the source unchanged instead of translating it.

Interesting ideas

  • I1. Adding a cheap sentence-level classification head on the shared encoder gives the generator an extra signal about the input's variety at almost no architectural cost.
  • I2. They filtered the corpus by keeping only lines where a dialect classifier's prediction matched the dialect expected from the video creator's background. Metadata and a model act as two independent witnesses that must agree.
  • I3. The error analysis shows that a wrong "already standard" judgment silently turns translation into copying. This failure is invisible in average BLEU but easy to count.
  • I4. The target side was built by pivoting through English with an LLM and then auditing a small random sample by hand. This is a pragmatic way to bootstrap parallel data while still estimating its quality.
  • I5. Subtitle chunks were rebuilt into sentence-like units by merging close segments that lack final punctuation and dropping fragments under four words.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): gparse could carry an auxiliary head predicting the source treebank or genre, which might help the parser adapt its attachments to the register of the input.
  • I2 (learn): silver sentences could be admitted only when witness-parser consensus also agrees with document metadata such as source or genre, discarding mismatches as likely noise.
  • I3 (learn): evaluation could track an explicit identity-output rate as a negative control, e.g. how often a dialect-conversion or repair step returns its input untouched.
  • I4 (learn): LLM teacher tiers could follow the same pattern, generating labels at scale and reporting a measured error rate from a fixed human-checked sample before the data is trusted.
  • I5 (tokens): the sentence splitter could use similar timing- or punctuation-based merge rules and minimum-length filters when ingesting transcripts or subtitle text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ManchesterManchester · United Kingdom93 works · 45,138 citations🎓King Abdulaziz UniversityJeddah · Saudi Arabia33 works · 508 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 3,066 works

Cite

Cite as Abdullah Khered, Youcef Benkhedda and Riza Batista-Navarro. 2025. A Multi-Task Learning Approach to Dialectal Arabic Identification and Translation to Modern Standard Arabic. Proceedings of the First Workshop on Advancing NLP for Low-Resource Languages.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{khered2025multi,
  title = {{A Multi-Task Learning Approach to Dialectal Arabic Identification and Translation to Modern Standard Arabic}},
  author = {Abdullah Khered, Youcef Benkhedda and Riza Batista-Navarro},
  booktitle = {Proceedings of the First Workshop on Advancing NLP for Low-Resource Languages},
  year = {2025},
  doi = {10.26615/978-954-452-100-4-004},
  url = {https://doi.org/10.26615/978-954-452-100-4-004}
}

References

35 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Abdellah El Mekki, Abdelkader El Mahdaouy, Kabil Essefar, Nabil El Mamoun, Ismail Berrada, and Ahmed Khoumsi. 2021. BERT-based multi-task model for country and province level MSA and dialectal Arabic identification. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 271–275, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
  3. Muhammad Abdul-Mageed, Chiyu Zhang, Abdel-Rahim A. Elmadany, Arun Rajendran, and Lyle H. Ungar. 2019. Dianet: BERT and hierarchical attention multi-task learning of fine-grained dialect. CoRR, abs/1910.14243. ↗
  4. Salman Elgamal, Ossama Obeid, Mhd Kabbani, Go Inoue, and Nizar Habash. 2024. Arabic diacritics in the wild: Exploiting opportunities for improved diacritization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14815–14829, Bangkok, Thailand. Association for Computational Linguistics.
  5. Mansour Al Sulaiman, Abdullah M. Moussa, Sherif Abdou, Hebah Elgibreen, Mohammed Faisal, and Mohsen Rashwan. 2022. Semantic textual similarity for modern standard and dialectal arabic using transfer learning. PLOS ONE, 17(8):1–14. find
  6. AbdelRahim Elmadany, El Moatez Billah Nagoudi, and Muhammad Abdul-Mageed. 2023. Octopus: A multitask model and toolkit for Arabic natural language generation. In Proceedings of ArabicNLP 2023, pages 232–243, Singapore (Hybrid). Association for Computational Linguistics.
  7. Ali Alkhathlan and Khalid Alomar. 2024. Armt-tnn: Enhancing natural language understanding performance through hard parameter multitask learning in arabic. Int. J. Know.-Based Intell. Eng. Syst., 28(3):483495. ↗
  8. Erick Mendez Guzman, Viktor Schlegel, and Riza Batista-Navarro. 2024. Towards explainable multi-label text classification: A multi-task rationalisation framework for identifying indicators of forced labour. In Proceedings of the Third Workshop on NLP for Positive Impact, pages 98–112, Miami, Florida, USA. Association for Computational Linguistics. find
  9. Sawsan Alqahtani, Ajay Mishra, and Mona Diab. 2020. A multitask learning approach for diacritic restoration. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8238–8247, Online. Association for Computational Linguistics.
  10. Moussa Kamal Eddine, Nadi Tomeh, Nizar Habash, Joseph Le Roux, and Michalis Vazirgiannis. 2022. AraBART: a pretrained Arabic sequence-to-sequence model for abstractive summarization. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 31–42, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  11. Meshrif Alruily. 2020. Issues of dialectal saudi twitter corpus. The International Arab Journal of Information Technology, 17:367–374. ↗
  12. Laith Baniata, Seyoung Park, and Seong-Bae Park. 2018a. A multitask-based neural machine translation model with part-of-speech tags integration for arabic dialects. Applied Sciences, 8:2502. ↗
  13. Abdullah Khered, Ingy Abdelhalim, Nadine Abdelhalim, Ahmed Soliman, and Riza Theresa Batista-Navarro. 2023. Unimanc at nadi 2023 shared task: A comparison of various t5-based models for translating arabic dialectical text to modern standard arabic. In Proceedings of ArabicNLP 2023, pages 658–664.
  14. Hamdy Mubarak. 2018. Dial2msa: A tweets corpus for converting dialectal arabic to modern standard arabic. In OSACT 3: The 3rd Workshop on Open-Source Arabic Corpora and Processing Tools, pages 49–53. find
  15. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. ↗
  16. Abdullah Khered, Ingy Yasser Hassan Abdou Abdelhalim, and Riza Batista-Navarro. 2022. Building an ensemble of transformer models for Arabic dialect classification and sentiment analysis. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 479–484, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  17. Hai Pham, Young Jin Kim, Subhabrata Mukherjee, David P. Woodruff, Barnab,s Pczos, and Hany Hassan Awadalla. 2023. Task-based moe for multitask multilingual machine translation. CoRR, abs/2308.15772. ↗
  18. Abdullah Khered, Youcef Benkhedda, and Riza Batista-Navarro. 2025. Dial2MSA-verified: A multi-dialect Arabic social media dataset for neural machine translation to Modern Standard Arabic. In Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4), pages 50–62, Abu Dhabi, UAE. Association for Computational Linguistics.
  19. Maja Popović. 2017. chrF++: words helping character n-grams. In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics. ↗
  20. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. ↗
  21. Abhishek Kumar, Asif Ekbal, Daisuke Kawahra, and Sadao Kurohashi. 2019. Emotion helps sentiment: A multi-task model for sentiment and emotion analysis. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1–8. find
  22. Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. 2020. Pre-trained models for natural language processing: A survey. Science China Technological Sciences, 63:1872– 1897. ↗
  23. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics. ↗
  24. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. ↗
  25. Fatma Mallek, Billal Belainine, and Fatiha Sadat. 2017. Arabic social media analysis and translation. Procedia Computer Science, 117:298–303. Arabic Computational Linguistics. find
  26. Fatiha Sadat, Farnazeh Kazemi, and Atefeh Farzindar. 2014. Automatic identification of arabic dialects in social media. SoMeRA ’14, page 3540, New York, NY, USA. Association for Computing Machinery. find
  27. Mamta, Asif Ekbal, and Pushpak Bhattacharyya. 2022. Exploring multi-lingual, multi-task, and adversarial learning for low-resource sentiment analysis. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 21(5). ↗
  28. Saudi Sadiq. 2025. Evaluating english-arabic translation: Human translators vs. google translate and chatgpt. Journal of Languages and Translation, 12(1):67– 95. Karima. ↗
  29. Meftouh, Salima Harrat, and Kamel Smaïli. 2018. PADIC: extension and new experiments. In 7th International Conference on Advanced Technologies ICAT, Antalya, Turkey. find
  30. Wael Salloum, Heba Elfardy, Linda Alamir-Salloum, Nizar Habash, and Mona Diab. 2014. Sentence level dialect identification for machine translation system selection. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 772–778, Baltimore, Maryland. Association for Computational Linguistics. find
  31. Youness Moukafih, Nada Sbihi, Mounir Ghogho, and Kamel Smaili. 2021. Improving machine translation ofnbsp;arabic dialects through multi-task learning. In AIxIA 2021 Advances in Artificial Intelligence: 20th International Conference of the Italian Association for Artificial Intelligence, Virtual Event, December 13, 2021, Revised Selected Papers, page 580590, Berlin, Heidelberg. Springer-Verlag. find
  32. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc. ↗
  33. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  34. Poorya Zaremoodi, Wray Buntine, and Gholamreza Haffari. 2018. Adaptive knowledge sharing in multi-task learning: Improving low-resource neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 656–661, Melbourne, Australia. Association for Computational Linguistics. find
  35. Yu Zhang and Qiang Yang. 2017. An overview of multi-task learning. National Science Review, 5(1):30–43. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/khered-2025-multi-task-learning-approach-dialectal/