Elamur Research⌕
☰
home / papers / 2025 / Dialectal Coverage And Generalization in Arabic Speech Recognition

Dialectal Coverage And Generalization in Arabic Speech Recognition

Paper · 2025 · Amirbek Djanibekov, Hawau Olamide Toyin, Raghad Alshalan et al.

Summary

Most Arabic ASR systems handle Modern Standard Arabic (MSA) and a few well-resourced dialects. They do poorly on the many other regional varieties and on speech mixing Arabic with English or French. The authors start from ArTST, an Arabic-only model built on the SpeechT5 encoder-decoder design. They re-pretrain it in three variants: v1 uses MSA only, v2 mixes MSA with dialectal speech and text, and v3 adds multilingual data. The pre-training data comes from 17 Arabic-speaking countries.

They then compare two fine-tuning strategies, dialect-specific and joint multi-dialectal, and also test zero-shot transfer to held-out dialects and performance on code-switched speech. Their findings:

  • Broader dialectal pre-training helps low-resource and unseen dialects without hurting MSA. The v2 model reaches state-of-the-art WER on MGB2.
  • Joint fine-tuning benefits weaker dialects, while high-resource dialects do better with their own dedicated fine-tuning.
  • Multilingual training sharply reduces errors on code-switched speech, but it degrades dialect accuracy.

All checkpoints and scripts are open. This matters because it gives open models covering far more of spoken Arabic and shows the trade-offs between these strategies.

Key points

  • Three pre-training variants of ArTST:
    • v1: MSA only (MGB2).
    • v2: MSA plus dialects, drawing on sources such as QASR, MGB3, MGB5, SADA, MASC and Common Voice, plus MADAR/NADI text.
    • v3: multilingual, adding Common Voice EN/FR/SP and code-switched sets.
  • The joint multi-dialectal model pools the train sets of 12 dialects. It matches or beats per-dialect fine-tuning on average and is best on held-out dialects.
  • Results on MGB2: the v2-based model scores 12.5% WER, while the multilingual v3 model scores 13.0%, so MSA loses only 0.5 points to language interference. Dialects, however, lose 4–16 points absolute WER.
  • On code-switched test sets, the best system starts from the joint multilingual v3 checkpoint and then fine-tunes on the target set:
    • ArzEn (Egyptian-English): 27.43% WER
    • TunSwitch (Tunisian-French): 36.66% WER
    • Mixat (Emirati-English): 21.66% WER
  • Using dialect ID during decoding helps, but imposing a fixed dialect ID is worse than letting the model infer it. The authors found that MASC's Syrian dev/test sets are actually MSA.

Interesting ideas

  • I1. Let the model infer the variety label. Predicting the variety tag beats supplying it, because labels are coarse and often wrong, so the model should be free to choose.
  • I2. Audit dataset labels by hand. A quick manual check revealed that a "dialect" test set was really standard language, which would have silently distorted the conclusions.
  • I3. Score against several valid references. Multi-reference scoring (four references for MGB3/MGB5) makes the metric fair when no single spelling or surface form is correct.
  • I4. Adapt in stages. First adapt from a strong general checkpoint (an MSA or joint model), then fine-tune on the small target set; the gains are largest for low-resource varieties.
  • I5. Measure interference on the core benchmark. Whenever coverage is broadened, re-check the main benchmark and each sub-variety separately, since a small loss on the headline number can hide large losses elsewhere.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): When converting or parsing text from different UD treebank dialects, gparse could predict the source-style tag itself rather than trusting the treebank's declared one.
  • I2 (learn): Spot-check silver data and per-domain test sets to confirm their genre or register labels before reporting per-domain scores.
  • I3 (back): Evaluate graph → text regeneration and translation against several accepted paraphrases rather than a single string.
  • I4 (learn): Train the parser on the full silver corpus first, then adapt it to small niche domains such as fables or maths word problems.
  • I5 (learn): Treat per-domain regression checks as negative controls whenever new languages, genres or teacher data are added to training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksSpeech & spoken language · 480 works

Cite

Cite as Amirbek Djanibekov, Hawau Olamide Toyin, Raghad Alshalan, Abdullah Alatir and Hanan Aldarmaki. 2025. Dialectal Coverage And Generalization in Arabic Speech Recognition. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.1427

@inproceedings{djanibekov2025dialectal,
  title = {{Dialectal Coverage And Generalization in Arabic Speech Recognition}},
  author = {Amirbek Djanibekov, Hawau Olamide Toyin, Raghad Alshalan, Abdullah Alatir and Hanan Aldarmaki},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1427},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1427}
}

References

31 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Amine Ben Abdallah, Ata Kabboudi, Amir Kanoun, and Salah Zaiem. 2024. Leveraging data collection and unsupervised learning for code-switched tunisian arabic automatic speech recognition. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12607–12611. IEEE.
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, El Moatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. NADI 2023: The fourth nuanced Arabic dialect identification shared task. In Proceedings of ArabicNLP 2023, Singapore (Hybrid). Association for Computational Linguistics.
  3. Maryam Khalifa Al Ali and Hanan Aldarmaki. 2024. Mixat: A data set of bilingual emirati-English speech. In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024. ↗
  4. Mohammad Al-Fetyani, Muhammad Al-Barham, Gheith Abandah, Adham Alsharkawi, and Maha Dawas. 2021. Masc: Massive arabic speech corpus. find
  5. Rania Al-Sabbagh. 2024. Arzen-multigenre: An aligned parallel dataset of egyptian arabic song lyrics, novels, and subtitles, with english translations. Data in Brief. ↗
  6. Sadeen Alharbi, Areeb Alowisheq, Zoltán Tüske, Kareem Darwish, Abdullah Alrajeh, Abdulmajeed Al-rowithi, Aljawharah Bin Tamran, Asma Ibrahim, Raghad Aloraini, Raneem Alnajim, Ranya Alkahtani, Renad Almuasaad, Sara Alrasheed, Shaykhah Alsubaie, and Yaser Alonaizan. 2024. Sada: Saudi audio dataset for arabic. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10286–10290. find
  7. Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang. 2016. The mgb-2 challenge: Arabic multi-dialect broadcast media recognition. In 2016 IEEE Spoken Language Technology Workshop (SLT). IEEE.
  8. Ahmed Ali, Suwon Shon, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, James Glass, Steve Renals, and Khalid Choukri. 2019. The mgb-5 challenge: Recognition and dialect identification of dialectal arabic speech. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE. find
  9. Ahmed Ali, Stephan Vogel, and Steve Renals. 2017. Speech recognition challenge in the wild: Arabic mgb-3. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE.
  10. Khalid Almeman, Mark Lee, and Ali Abdulrahman Almiman. 2013. Multi dialect arabic speech parallel corpora. In 2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA). find
  11. Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, and Furu Wei. 2022. SpeechT5: Unified-modal encoder-decoder pre-training for spoken language processing. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5723–5738, Dublin, Ireland. Association for Computational Linguistics. ↗
  12. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4218–4222. ↗
  13. Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, et al. 2022. Xls-r: Self-supervised cross-lingual speech representation learning at scale. In Proc. Interspeech 2022, pages 2278–2282.
  14. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems.
  15. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). find
  16. Amira Dhouib, Achraf Othman, Oussama El Ghoul, Mohamed Koutheair Khribi, and Aisha Al Sinani. 2022. Arabic automatic speech recognition: a systematic literature review. Applied Sciences, 12(17):8898. ↗
  17. Nawar Halabi et al. 2016. Arabic speech corpus. Oxford Text Archive Core Collection. find
  18. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing. ↗
  19. Amir Hussein, Shinji Watanabe, and Ahmed Ali. 2022. Arabic speech recognition by end-to-end, modular systems and human. Computer Speech & Language, 71:101272. ↗
  20. Karima Kadaoui, Maryam Al Ali, Hawau Olamide Toyin, Ibrahim Mohammed, and Hanan Aldarmaki. 2024. PolyWER: A holistic evaluation framework for code-switched speech recognition. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6144–6153, Miami, Florida, USA. Association for Computational Linguistics. find
  21. Ajinkya Kulkarni, Atharva Kulkarni, Sara Abedal-mon’em Mohammad Shatnawi, and Hanan Aldarmaki. 2023. Clartts: An open-source classical arabic text-to-speech corpus. In 2023 INTERSPEECH.
  22. Salima Mdhaffar, Fethi Bougares, Renato de Mori, Salah Zaiem, Mirco Ravanelli, and Yannick Estève. 2024. TARIC-SLU: A Tunisian benchmark dataset for spoken language understanding. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15606–15616, Torino, Italia. ELRA and ICCL. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR. find
  23. Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. In Proc. Interspeech 2019, pages 3465–3469. find
  24. Barbara Schuppler, Martine Adda-Decker, Catia Cucchiarini, and Rudolf Muhr. 2024. An introduction to pluricentric languages in speech science and technology. Speech Communication, 156:103007. ↗
  25. Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, and Hanan Aldarmaki. 2023. ArTST: Arabic text and speech transformer. In Proceedings of ArabicNLP 2023, pages 41–51, Singapore (Hybrid). Association for Computational Linguistics.
  26. Abdul Waheed, Karima Kadaoui, and Muhammad Abdul-Mageed. 2024. To distill or not to distill? on the robustness of robust knowledge distillation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12603–12621.
  27. Abdul Waheed, Bashar Talafha, Peter Sullivan, Abdel-Rahim Elmadany, and Muhammad Abdul-Mageed. 2023. VoxArabica: A robust dialect-aware Arabic speech recognition system. In Proceedings of ArabicNLP 2023, pages 441–449, Singapore (Hybrid). Association for Computational Linguistics.
  28. Chengyi Wang, Yu Wu, Yao Qian, Kenichi Kumatani, Shujie Liu, Furu Wei, Michael Zeng, and Xuedong Huang. 2021. Unispeech: Unified speech representation learning with labeled and unlabeled data. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, Proceedings of Machine Learning Research. PMLR. ↗
  29. Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. 2021. QASR: QCRI aljazeera speech resource a large scale annotated Arabic speech corpus. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2274–2285, Online. Association for Computational Linguistics. ↗
  30. Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, et al. 2024. Scaling speech technology to 1,000+ languages. Journal of Machine Learning Research. ↗
  31. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/djanibekov-2025-dialectal-coverage-generalization-arabic-speech/