Elamur Research⌕
☰
home / papers / 2026 / Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis

Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis

Paper · 2026 · Yushen Chen, Junzhe Liu, Yujie Tu et al.

Summary

Arabic has roughly 30 spoken varieties. Before this work, no open-source text-to-speech system covered them in a single model. Three obstacles stood in the way: large lexical and phonological differences between dialects, little audio clean enough for synthesis, and no shared benchmark for comparing systems.

Habibi addresses all three. The authors take public ASR corpora, plus some in-house and manually transcribed material, and turn them into about 1,857 hours of TTS training audio spanning a dozen regional dialects. Cleaning relies on speaking-rate filters, source-separation denoising and merging of short segments.

Training starts from an F5-TTS checkpoint and follows a two-stage curriculum. The model is first fine-tuned on Modern Standard Arabic and then on dialectal data tagged with region tokens. No diacritized text is needed.

The authors also release a benchmark of more than 11,000 utterances in 7 dialect subsets, with transcripts checked by hand. On this benchmark, one unified model equals or beats models specialized for a single dialect. It also holds its own against ElevenLabs' Eleven v3 (alpha). The release gives long-tail dialect speech research an open baseline that others can reproduce.

Key points

  • The training data totals 1,857.3 h and 879,339 utterances. Sources include MASC, MGB-2/3/5, SADA, FLEURS, the Omnilingual ASR Corpus, UAE-100K, Darija-S2T, DarijaTTS-clean, Jordan-Audio and in-house sets. Expanding the data from D1 (1,635 h) to D2 added OMN, TUN, SDN and LBY.
  • Each of the benchmark's seven subsets (Egyptian, Moroccan, Algerian, Iraqi, Saudi, Emirati and Modern Standard Arabic) holds between 1,000 and 3,000 clips. Clips last 3–12 s, use Arabic script only, and each speaker has a second utterance available to serve as the reference prompt. The benchmark is kept strictly separate from the training data.
  • The speaking-rate (characters per second) filter uses a lower bound between 4 and 10 and an upper bound between 15 and 25. Noisier sources get stricter thresholds.
  • Raw and denoised audio are mixed during training: the denoised version is sampled with probability 0.618. Data quality mattered more than quantity. Adding noisy or pseudo-clean MASC audio made EGY results worse, while moving from D1 to D2 lowered average WER-O from 19.28 to 19.10.
  • The ablations took about 8,000 H100 GPU hours across more than 30 configurations. Every checkpoint, the code and the benchmark are released openly.

Interesting ideas

  • I1. Standard-first curriculum. The model learns the most regular, best-resourced variety before the noisy dialectal data, which helps it converge stably and reach better final quality.
  • I2. Simple ratio filter for corrupted pairs. A characters-per-second check catches missing labels or broken audio with little effort, and its strictness is tuned to how dirty each source looks.
  • I3. Variety tags with an "unknown" fallback. Region tokens make the dialect distributions easier to tell apart during training. At inference the model still works when the dialect is unknown or the tag is missing, though matching the training format gives the best results.
  • I4. Two kinds of judges for evaluation. Each output is scored by both a general multilingual ASR and a dialect-specialized ASR, so the result does not depend on one recognizer's blind spots.
  • I5. Clean and enhanced versions mixed by probability. Original and denoised copies are sampled stochastically, so the model sees both conditions instead of only one.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): train parsers on edited, standard English treebanks first, then fine-tune on web, spoken or informal UD data, rather than mixing everything from the beginning.
  • I2 (learn): screen silver sentences with cheap ratio checks, such as tokens per character or tree depth against length, and set tighter limits for noisier sources.
  • I3 (tree): give gparse an input token for genre or English variety, plus an "unknown" option, so it can learn register-specific attachment habits without becoming dependent on the tag.
  • I4 (back): check generated text by re-parsing it with two different parsers (perceptron and gparse) and treating disagreement as a warning sign.
  • I5 (tokens): train the tagger on raw and normalized versions of the same noisy text (spelling-fixed, de-cased), so it stays robust whether or not normalization is applied.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China149 works · 16,161 citations🎓University of Chinese Academy of SciencesBeijing · China147 works · 8,480 citations🎓Shanghai Innovation InstituteShanghai · China24 works · 262 citations🏢KuaishouBeijing · China15 works · 95 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksSpeech & spoken language · 480 works

Cite

Cite as Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Chunyu Qiang, Chen Zhang, Kai Yu and Xie Chen. 2026. Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis. arXiv:2601.13802.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2601.13802 · arXiv 2601.13802

@misc{chen2026habibi,
  title = {{Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesis}},
  author = {Yushen Chen, Junzhe Liu, Yujie Tu, Zhikang Niu, Yuzhe Liang, Chunyu Qiang, Chen Zhang, Kai Yu and Xie Chen},
  year = {2026},
  doi = {10.48550/arxiv.2601.13802},
  eprint = {2601.13802},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2601.13802}
}

References

54 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Maryam Khalifa Al Ali and Hanan Aldarmaki. 2024. Mixat: A data set of bilingual emirati-English speech. In Proc. LREC-COLING. 222–226. ↗
  2. Mohammad Al-Fetyani, Muhammad Al-Barham, Gheith Abandah, Adham Alsharkawi, and Maha Dawas. 2023. MASC: Massive arabic speech corpus. In Proc. SLT. IEEE, 1006–1013. find
  3. Sadeen Alharbi, Areeb Alowisheq, Zoltán Tüske, Kareem Darwish, Abdullah Alrajeh, Abdulmajeed Alrowithi, Aljawharah Bin Tamran, Asma Ibrahim, Raghad Aloraini, Raneem Alnajim, et al. 2024. SADA: Saudi audio dataset for arabic. In Proc. ICASSP. IEEE, 10286–10290. find
  4. Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang. 2016. The MGB-2 challenge: Arabic multi-dialect broadcast media recognition. In Proc. SLT. IEEE, 279–284.
  5. Ahmed Ali, Suwon Shon, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, James Glass, Steve Renals, and Khalid Choukri. 2019. The MGB-5 challenge: Recognition and dialect identification of dialectal arabic speech. In Proc. ASRU. IEEE, 1026–1033. find
  6. Ahmed Ali, Stephan Vogel, and Steve Renals. 2017. Speech recognition challenge in the wild: Arabic MGB-3. In Proc. ASRU. IEEE, 316–322.
  7. Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, et al. 2024. Seed-TTS: A Family of High-Quality Versatile Speech Generation Models. arXiv preprint arXiv:2406.02430 (2024). ↗
  8. James Betker. 2023. Better speech synthesis through scaling. arXiv preprint arXiv:2305.07243 (2023). ↗
  9. Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, et al. 2024. XTTS: a massively multilingual zero-shot text-to-speech model. arXiv preprint arXiv:2406.04904 (2024). ↗
  10. Wenxi Chen, Xinsheng Wang, Ruiqi Yan, Yushen Chen, Zhikang Niu, Ziyang Ma, Xiquan Li, Yuzhe Liang, Hanlin Wen, Shunshun Yin, et al. 2025. SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization. arXiv preprint arXiv:2510.16841 (2025). ↗
  11. Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. 2025. F5-TTS: A fairytaler that fakes fluent and faithful speech with flow matching. In Proc. ACL. 6255–6271.
  12. Zhengyang Chen, Sanyuan Chen, Yu Wu, Yao Qian, Chengyi Wang, Shujie Liu, Yanmin Qian, and Michael Zeng. 2022. Large-scale self-supervised speech representation learning for automatic speaker verification. In Proc. ICASSP. IEEE, 6147–6151. ↗
  13. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2023. FLEURS: Few-shot learning evaluation of universal representations of speech. In Proc. SLT. IEEE, 798–805.
  14. Amirbek Djanibekov, Hawau Olamide Toyin, Raghad Alshalan, Abdullah Alatir, and Hanan Aldarmaki. 2025. Dialectal coverage and generalization in Arabic speech recognition. In Proc. ACL. 29490–29502. ↗
  15. Khai Doan, Abdul Waheed, and Muhammad Abdul-Mageed. 2024. Towards zero-shot text-to-speech for Arabic dialects. In Proc. ArabicNLP. 123–129.
  16. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, et al. 2024. CosyVoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407 (2024). ↗
  17. Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, et al. 2024. E2 TTS: Embarrassingly easy fully non-autoregressive zero-shot tts. In Proc. SLT. IEEE, 682–689. find
  18. Pengchao Feng, Yao Xiao, Ziyang Ma, Zhikang Niu, Shuai Fan, Yao Li, Sheng Wang, and Xie Chen. 2025. Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesis. arXiv preprint arXiv:2512.18699 (2025). ↗
  19. Yuan Gao, Nobuyuki Morioka, Yu Zhang, and Nanxin Chen. 2023. E3 TTS: Easy end-to-end diffusion-based text to speech. In Proc. ASRU. IEEE, 1–8. find
  20. Hao-Han Guo, Kun Liu, Fei-Yu Shen, Yi-Chen Wu, Feng-Long Xie, Kun Xie, and Kai-Tuo Xu. 2024. FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applications. arXiv preprint arXiv:2409.03283 (2024). ↗
  21. Nawar Halabi. 2016. Modern standard Arabic phonetics for speech synthesis. Ph. D. Dissertation. University of Southampton. find
  22. Dongya Jia, Zhuo Chen, Jiawei Chen, Chenpeng Du, Jian Wu, Jian Cong, Xiaobin Zhuang, Chumin Li, Zhen Wei, Yuping Wang, et al. 2025. DiTAR: Diffusion transformer autoregressive modeling for speech generation. arXiv preprint arXiv:2502.03930 (2025). ↗
  23. Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al. 2024. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. arXiv preprint arXiv:2403.03100 (2024). ↗
  24. Ajinkya Kulkarni, Atharva Kulkarni, Sara Abedalmonem Mohammad Shatnawi, and Hanan Aldarmaki. 2023. ClArTTS: An open-source classical arabic text-to-speech corpus. arXiv preprint arXiv:2303.00069 (2023). ↗
  25. Imen Laouirine, Rami Kammoun, and Fethi Bougares. 2024. TunArTTS: Tunisian Arabic text-to-speech corpus. In Proc. LREC-COLING. 16879–16889.
  26. Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, et al. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. in Proc. NIPS 36 (2023), 14005–14034. find
  27. Shijia Liao, Yuxuan Wang, Tianyu Li, Yifan Cheng, Ruoyi Zhang, Rongzhi Zhou, and Yijin Xing. 2024. Fish-speech: Leveraging large language models for advanced multilingual text-to-speech synthesis. arXiv preprint arXiv:2411.01156 (2024). ↗
  28. Yi Luo and Jianwei Yu. 2023. Music source separation with band-split RNN. IEEE/ACM TASLP 31 (2023), 1893–1901. ↗
  29. Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, et al. 2024. Autoregressive Speech Synthesis without Vector Quantization. arXiv preprint arXiv:2407.08551 (2024). ↗
  30. Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. 2021. QASR: QCRI Aljazeera Speech Resource–A Large Scale Annotated Arabic Speech Corpus. arXiv preprint arXiv:2106.13000 (2021). ↗
  31. ASR Omnilingual, Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, et al. 2025. Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages. arXiv preprint arXiv:2511.09690 (2025). ↗
  32. Yi Ren, Chenxu Hu, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2020. Fastspeech 2: Fast and high-quality end-to-end text to speech. arXiv preprint arXiv:2006.04558 (2020). ↗
  33. Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2019. Fastspeech: Fast, robust and controllable text to speech. in Proc. NIPS 32 (2019). ↗
  34. Simon Rouard, Manu Orsini, Axel Roebel, Neil Zeghidour, and Alexandre Défossez. 2025. Continuous audio language models. arXiv preprint arXiv:2509.06926 (2025). ↗
  35. Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. UTMOS: Utokyo-sarulab system for voicemos challenge 2022. arXiv preprint arXiv:2204.02152 (2022). ↗
  36. Barbara Schuppler, Martine Adda-Decker, Catia Cucchiarini, and Rudolf Muhr. 2024. An introduction to pluricentric languages in speech science and technology. 103007 pages. ↗
  37. Yakun Song, Xiaobin Zhuang, Jiawei Chen, Zhikang Niu, Guanrou Yang, Chenpeng Du, Dongya Jia, Zhuo Chen, Yuping Wang, Yuxuan Wang, et al. 2025. DiSTAR: Diffusion over a Scalable Token Autoregressive Representation for Speech Generation. arXiv preprint arXiv:2510.12210 (2025). ↗
  38. Bashar Talafha, Hawau Olamide Toyin, Peter Sullivan, AbdelRahim A Elmadany, Abdurrahman Juma, Amirbek Djanibekov, Chiyu Zhang, Hamad Alshehhi, Hanan Aldarmaki, Mustafa Jarrar, et al. 2025. NADI 2025: The first multidialectal arabic speech processing shared task. In Proc. ArabicNLP: Shared Tasks. 720–733. ↗
  39. Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, and Hanan Aldarmaki. 2023. ArTST: Arabic text and speech transformer. arXiv preprint arXiv:2310.16621 (2023). ↗
  40. Hawau Olamide Toyin, Rufael Marew, Humaid Alblooshi, Samar M Magdy, and Hanan Aldarmaki. 2025. ArVoice: A Multi-Speaker Dataset for Arabic Speech Synthesis. arXiv preprint arXiv:2505.20506 (2025). ↗
  41. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111 (2023). ↗
  42. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, et al. 2025. Spark-TTS: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens. arXiv preprint arXiv:2503.01710 (2025). ↗
  43. Xiaopeng Wang, Chunyu Qiang, Ruibo Fu, Zhengqi Wen, Xuefei Liu, Yukun Liu, Yuzhe Liang, Kang Yin, Yuankun Xie, Heng Xie, et al. 2025. M3-TTS: Multi-modal dit alignment & mel-latent for zero-shot high-fidelity speech synthesis. arXiv preprint arXiv:2512.04720 (2025). ↗
  44. Yingzhi Wang, Anas Alhmoud, and Muhammad Alqurishi. 2024. Open universal arabic asr leaderboard. arXiv preprint arXiv:2412.13788 (2024). ↗
  45. Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, et al. 2017. Tacotron: Towards end-to-end speech synthesis. arXiv preprint arXiv:1703.10135 (2017). ↗
  46. Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. 2024. MaskGCT: Zero-shot text-to-speech with masked generative codec transformer. arXiv preprint arXiv:2409.00750 (2024). ↗
  47. Wikipedia. 2025. Varieties of Arabic. https://en.wikipedia.org/w/index.php? title=Varieties_of_Arabic&oldid=1329477423 [Online; accessed 5-January-2026]. ↗
  48. Zhen Ye, Xinfa Zhu, Chi-Min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi Dai, et al. 2025. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis. arXiv preprint arXiv:2502.04128 (2025). ↗
  49. Fan Yu, Tao Wang, You Wu, Lin Zhu, Wei Deng, Weisheng Han, Wenchao Wang, Lin Hu, Xiangyu Liang, Xiaodong He, et al. 2025. JoyVoice: Long-Context Conditioning for Anthropomorphic Multi-Speaker Conversational Synthesis. arXiv preprint arXiv:2512.19090 (2025). ↗
  50. Bowen Zhang, Congchao Guo, Geng Yang, Hang Yu, Haozhe Zhang, Heidi Lei, Jialong Mai, Junjie Yan, Kaiyue Yang, Mingqi Yang, et al. 2025. MiniMax-Speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder. arXiv preprint arXiv:2505.07916 (2025). ↗
  51. Zhiyuan Zhao, Lijian Lin, Ye Zhu, Kai Xie, Yunfei Liu, and Yu Li. 2026. LEMAS: Large A 150K-Hour Large-scale Extensible Multilingual Audio Suite with Generative Speech Models. arXiv preprint arXiv:2601.04233 (2026). ↗
  52. Siyi Zhou, Yiquan Zhou, Yi He, Xun Zhou, Jinchao Wang, Wei Deng, and Jingchen Shu. 2025. IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speech. arXiv preprint arXiv:2506.21619 (2025). ↗
  53. Han Zhu, Wei Kang, Zengwei Yao, Liyong Guo, Fangjun Kuang, Zhaoqing Li, Weiji Zhuang, Long Lin, and Daniel Povey. 2025. ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matching. arXiv preprint arXiv:2506.13053 (2025). ↗
  54. Jianheng Zhuo, Yifan Yang, Yiwen Shao, Yong Xu, Dong Yu, Kai Yu, and Xie Chen. 2025. VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretraining. arXiv preprint arXiv:2505.21527 (2025). ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2026-habibi-laying-open-source-foundation/