Elamur Research⌕
☰
home / papers / 2025 / Open Automatic Speech Recognition Models for Classical and Modern Standard Arabic

Open Automatic Speech Recognition Models for Classical and Modern Standard Arabic

Summary

Arabic speech recognition still has few strong open models. Most existing work covers Modern Standard Arabic (MSA) and largely ignores Classical Arabic (CA), the fully vowelized language of the Quran. Short vowels are usually left out of written Arabic, so systems that drop diacritics produce ambiguous text. The NVIDIA authors built an open preprocessing pipeline for Arabic audio and text in NeMo-speech-data-processor. It fixes the character inventory, normalizes Unicode and numerals, and filters noisy or mislabeled clips. With it they trained two FastConformer models using hybrid RNN-T/CTC decoding on openly licensed corpora: MASC, MCV17.0, FLEURS and EveryAyah. The first model handles MSA and outputs punctuation. The second is described as the first public model covering both MSA and CA, and it also predicts diacritics. The authors report state-of-the-art results for MSA, including blind tests on MGB2 and MediaSpeech, and the best diacritized CA accuracy. Checkpoints and recipes are released under a commercial-friendly license, which gives others a reproducible baseline for Arabic and for diacritized ASR.

Key points

  • The two checkpoints on HuggingFace are stt_ar_fastconformer_hybrid_large_pc_v1.0 (MSA, with punctuation) and ..._pcd_v1.0 (MSA+CA, with punctuation and diacritics).
  • The output alphabet has 36 Arabic characters: 28 base letters, six Hamza forms, Ta' Marbuta and Alif Maksura. Adding Fathah, Kasrah, Dammah, Sukun, Tanween and Shaddah brings it to 44 symbols. Only three punctuation marks are kept.
  • The training corpora are MASC (646h clean + 855h noisy raw), MCV17.0 Arabic, FLEURS Arabic and EveryAyah, with EveryAyah used only for the CA model. MGB2 and MediaSpeech are used solely as blind test sets.
  • Filtering with a pretrained model's WER/CER and with word-rate and character-rate checks recovered roughly 120h of extra usable clean MASC audio and 40h of noisy audio.
  • The EveryAyah test set had train/test text overlap. Removing it shrank the test set from 100h to 23h, and results are given on both versions. Training used dropout 0.1, SpecAugment with 4 frequency masks and 20 time masks, and a CTC weight of 0.3.

Interesting ideas

  • I1. Leakage between train and test is checked only after stripping diacritics and punctuation, because the same sentence can reappear with different markup or a different speaker. — For Mova (learn): Before scoring, deduplicate treebank and silver splits on a normalized form (lowercased, punctuation and casing removed), so a reworded copy of a training sentence cannot inflate LAS.
  • I2. An already trained model screens new training data, and clips it transcribes very badly are treated as likely label noise rather than hard examples. — For Mova (learn): Let the witness parsers score incoming silver sentences and drop those where every witness scores very poorly against the proposed tree, keeping noisy text out of training.
  • I3. The symbol inventory is closed and explicit: any sample containing an out-of-alphabet character is discarded, and NFKC is applied first so positional letter forms and stacked marks become one canonical sequence. — For Mova (tokens): Declare a fixed character set for the en tokenizer and canonicalize Unicode (ligatures, quote variants, combining marks) before tagging, so that rare glyphs do not create spurious lemmas.
  • I4. Each benchmark is scored only after the model output is converted to that benchmark's own normalization (letter folding for MASC and MediaSpeech), and the models are never trained on the evaluation corpus. — For Mova (tree): Evaluate gparse blind on treebanks it never trained on, after converting its output into the target UD dialect, so that the score reflects parsing quality and not annotation-convention mismatch.
  • I5. The release keeps two variants of the same model, one with plain output and one that also predicts diacritics, so users can pick the richer layer when they need it without losing the simpler one. — For Mova (back): Generation could likewise offer a plain surface form and an annotated variant with explicit morphology or role marks, checking that both come from the same meaning graph.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

2 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2507.13977 · arXiv 2507.13977

@misc{grigoryan2025open,
  title = {{Open Automatic Speech Recognition Models for Classical and Modern Standard Arabic}},
  author = {Grigoryan, Lilit and Karpov, Nikolay and Albasiri, Enas and Lavrukhin, Vitaly and Ginsburg, Boris},
  year = {2025},
  doi = {10.48550/arxiv.2507.13977},
  eprint = {2507.13977},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2507.13977}
}

References

46 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. N. Corporation, “Nvidia nemo: A toolkit for conversational ai,” 2020. [Online]. Available: https://developer.nvidia.com/nvidia-nemo ↗
  2. M. Nofal, E. Reheem, H. Henawy, and N. Kader, “The development of acoustic models for command and control arabic speech recognition system,” 01 2004. find
  3. F. Elmisery, A. Khalil, A. Salama, and H. Hammed, “A fpga-based hmm for a discrete arabic speech recognition system,” pp. 322–325, 2003. find
  4. B. Hocine, R. Djemili, M. Bedda, and C. Snani, “New hybrid system (supervised classifier/hmm) for isolated arabic speech recognition,” 01 2006, pp. 1264 – 1269. find
  5. A. Ali, Y. Zhang, P. Cardinal, N. Dahak, S. Vogel, and J. Glass, “A complete kaldi recipe for building arabic speech recognition systems,” 2014 IEEE Workshop on Spoken Language Technology, SLT 2014 - Proceedings, pp. 525–529, 04 2015. find
  6. A. E. Kourd and K. E. Kourd, “Arabic isolated word speaker dependent recognition system,” Journal of Advances in Mathematics and Computer Science, vol. 14, no. 1, p. 1–15, Jan. 2016. find
  7. P. Cardinal, A. Ali, N. Dehak, Y. Zhang, T. Hanai, J. Glass, and S. Vogel, “Recent advances in asr applied to an arabic transcription system for al-jazeera,” Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pp. 2088–2092, 01 2014. find
  8. T. AlHanai, W.-N. Hsu, and J. Glass, “Development of the mit asr system for the 2016 arabic multi-genre broadcast challenge,” in 2016 IEEE Spoken Language Technology Workshop (SLT), 2016, pp. 299–304. find
  9. A. Ouisaadane and S. Said, “A comparative study for arabic speech recognition system in noisy environments,” International Journal of Speech Technology, vol. 24, pp. 1–10, 09 2021. find
  10. A. Masmoudi, F. Bougares, M. Ellouze, Y. Estève, and L. Belguith, “Automatic speech recognition system for tunisian dialect,” Language Resources and Evaluation, vol. 52, 03 2018. find
  11. M. Elmahdy, M. Hasegawa-Johnson, and E. Mustafawi, “Development of a TV broadcasts speech recognition system for qatari Arabic,” in Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), 2014, pp. 3057–3061. find
  12. A. Ahmed, Y. Hifny, K. Shaalan, and S. Toral, End-to-End Lexicon Free Arabic Speech Recognition Using Recurrent Neural Networks, 11 2018, pp. 231–248. find
  13. N. Zerari, S. Abdelhamid, H. Bouzgou, and C. Raymond, “Bidirectional deep architecture for arabic speech recognition,” Open Computer Science, vol. 9, no. 1, pp. 92–102, 2019. find
  14. Y. Belinkov, A. Ali, and J. Glass, “Analyzing phonetic and graphemic representations in end-to-end automatic speech recognition,” 2020. find
  15. H. A. Alsayadi, A. A. Abdelhamid, I. Hegazy, and Z. T. Fayed, “Arabic speech recognition using end-to-end deep learning,” IET Signal Process., vol. 15, pp. 521–534, 2021. find
  16. S. A. Chowdhury, A. Hussein, A. Abdelali, and A. Ali, “Towards One Model to Rule All: Multilingual Strategy for Dialectal Code-Switching Arabic ASR,” in Proc. Interspeech 2021, 2021, pp. 2466–2470. find our card
  17. L. Alkanhal, A. Alessa, E. Almahmoud, and R. Alaqil, “Aswat: Arabic audio dataset for automatic speech recognition using speech-representation learning,” in Proceedings of ArabicNLP 2023, 2023, pp. 120–127. find
  18. A. Hussein, S. Watanabe, and A. Ali, “Arabic speech recognition by end-to-end, modular systems and human,” 2021. find our card
  19. A. Ali, P. Bell, J. Glass, Y. Messaoui, H. Mubarak, S. Renals, and Y. Zhang, “The mgb-2 challenge: Arabic multi-dialect broadcast media recognition,” 12 2016, pp. 279–284. find our card
  20. V. Peddinti, D. Povey, and S. Khudanpur, “A time delay neural network architecture for efficient modeling of long temporal contexts,” in Interspeech 2015, 2015, pp. 3214–3218. find
  21. H. Sak, A. Senior, and F. Beaufays, “Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition,” 2014. find
  22. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd International Conference on Machine Learning, ser. ICML ’06. New York, NY, USA: Association for Computing Machinery, 2006, p. 369–376. find
  23. A. Hussein, S. Watanabe, and A. Ali, “Arabic speech recognition by end-to-end, modular systems and human,” Computer Speech & Language, vol. 71, p. 101272, 2022. find our card
  24. A. Baevski, W.-N. Hsu, Q. Xu, A. Babu, J. Gu, and M. Auli, “data2vec: A general framework for self-supervised learning in speech, vision and language,” 2022. find
  25. R. Ardila, M. Branson, K. Davis et al., “Common voice: A massively-multilingual speech corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Conference, 2020, pp. 4218–4222. find our card
  26. B. Talafha, A. Waheed, and M. Abdul-Mageed, “N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition,” in Proc. INTER-SPEECH 2023, 2023, pp. 5092–5096. find our card
  27. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. find our card
  28. A. Conneau, M. Ma, S. Khanuja, Y. Zhang, V. Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “Fleurs: Few-shot learning evaluation of universal representations of speech,” 2022 IEEE Spoken Language Technology Workshop (SLT), pp. 798–805, 2022. find our card
  29. H. Toyin, A. Djanibekov, A. Kulkarni, and H. Aldarmaki, “ArTST: Arabic text and speech transformer,” in Proceedings of ArabicNLP 2023. Singapore (Hybrid): Association for Computational Linguistics, Dec. 2023, pp. 41–51. find
  30. R. Kolobov, O. Okhapkina, A. P. Olga Omelchishina, R. Bedyakin, V. Moshkin, D. Menshikov, and N. Mikhaylovskiy, “Mediaspeech: Multilanguage asr benchmark and dataset,” 2021. find
  31. S. Kriman, S. Beliaev, B. Ginsburg, J. Huang, O. Kuchaiev, V. Lavrukhin, R. Leary, J. Li, and Y. Zhang, “Quartznet: Deep automatic speech recognition with 1d time-channel separable convolutions,” 2019. find
  32. M. Al-Fetyani, M. Al-Barham, G. Abandah, A. Alsharkawi, and M. Dawas, “Masc: Massive arabic speech corpus,” in 2022 IEEE Spoken Language Technology Workshop (SLT), 2023, pp. 1006–1013. find
  33. A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates, and A. Y. Ng, “Deep speech: Scaling up end-to-end speech recognition,” 2014. find our card
  34. D. Povey, A. Ghoshal, G. Boulianne, L. Zettlemoyer, V. Panayotov et al., “Kaldi speech recognition toolkit,” 2011. [Online]. Available: http://kaldi-asr.org/ ↗
  35. S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y. Unno, N. E. Y. Soplin, J. Heymann, M. Wiesner, N. Chen, A. Renduchintala, and T. Ochiai, “Espnet: End-to-end speech processing toolkit,” 2018. find
  36. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, ser. NIPS’17. Curran Associates Inc., 2017, p. 6000–6010. find
  37. A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang, “Conformer: Convolution-augmented transformer for speech recognition,” 2020. find our card
  38. D. Rekesh, N. R. Koluguri, S. Kriman, S. Majumdar, V. Noroozi, H. Huang, O. Hrinchuk, K. Puvvada, A. Kumar, J. Balam, and B. Ginsburg, “Fast conformer with linearly scalable attention for efficient speech recognition,” 2023. find
  39. “Tarteel ai’s everyayah dataset,” https://hf.co/datasets/tarteel-ai/everyayah, accessed: 2024-09-12. ↗
  40. N. Corporation, “Speech data processor (sdp) toolkit,” 2024. [Online]. Available: https://github.com/NVIDIA/NeMo-speech-data-processor ↗
  41. Y. Zhang, E. Bakhturina, and B. Ginsburg, “NeMo (Inverse) Text Normalization: From Development to Production,” in Proc. Interspeech 2021, 2021, pp. 4857–4859. find
  42. A. Graves, “Sequence transduction with recurrent neural networks,” 2012. find our card
  43. D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” in Interspeech 2019. ISCA, Sep. 2019. find our card
  44. N. Corporation, “Stt en fastconformer hybrid transducer-ctc large p&c,” 2020. [Online]. Available: https://hf.co/nvidia/stt en fastconformer hybrid large pc ↗
  45. ——, “Stt es fastconformer hybrid transducer-ctc large p&c,” 2020. [Online]. Available: https://hf.co/nvidia/stt es fastconformer hybrid large pc ↗
  46. “Seamlessm4t: Massively multilingual & multimodal machine translation,” 2023. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/grigoryan-2025-open-automatic-speech-recognition-models/