Elamur Research⌕
☰
home / papers / 2021 / Multi-Dialect Arabic Speech Recognition

Multi-Dialect Arabic Speech Recognition

Summary

The paper tackles automatic speech recognition for Arabic, which is hard for three reasons. Arabic has many dialects, open speech data is scarce, and transcripts often omit diacritics (vowel marks). The author makes two contributions.

The first is a multi-dialect training corpus of about 1,400 hours. It merges several public sources (Aldiri, KACST, Isolated Words, Arabic News Channel, KSU, MGB-2) with a newly recorded 70-hour read-speech set. All transcripts are mapped onto one shared character inventory, and audio is cleaned and resampled to a uniform format.

The second is an end-to-end acoustic model in the style of Deep Speech. Spectrogram input passes through a few convolutional layers and then a stack of bidirectional recurrent layers. The model is trained with CTC, and its output is decoded by beam search with a 4-gram language model.

Successive training refinements bring word error rate down to about 14%, which the author reports as better than earlier systems. The work shows that consolidating messy, heterogeneous sources can matter as much as the network design for an under-resourced language.

Key points

  • The corpus totals roughly 1,400 hours from about 450–500 speakers, with a female/male ratio of roughly 35/65. It covers MSA, Egyptian, Gulf, Levantine and North African speech, and MGB-2 alone contributes 1,200 hours.
  • The new 70-hour corpus reuses the phonetically balanced SAAVB prompt set of 1,033 phrases. Each speaker read 190 phrases, and any recording failing an automatic and analyst quality check was redone.
  • Diacritics were first limited to fatha, damma, kasra and shadda. Because these caused class imbalance, the final runs dropped all diacritics except hamza above. Transcripts with non-standard characters and segments with overlapping speech were removed.
  • Audio preprocessing used:
    • a 150 Hz high-pass filter;
    • 16 kHz, 16-bit mono LPCM;
    • segments of 2–30 s.
  • The model has three conv layers (32/32/96 filters, BatchNorm) and 4–5 bidirectional LSTM/GRU layers of width 768 or 1024 with 20% dropout. It uses clipped ReLU (capped at 20), early stopping, sorted training samples, CTC, and beam search with a 4-gram LM. WER improved step by step from 20.5 to 14.07.

Interesting ideas

  • I1. Instead of patching each source separately, the author defines one shared symbol inventory and maps every corpus onto it, discarding rare marks that only add imbalance. — For Mova (tree): When merging treebanks with different conventions, first fix a single target label and feature inventory, then drop or collapse rare idiosyncratic tags so they do not skew the perceptron parser or gparse.
  • I2. Problematic material such as overlapping speech and non-standard characters is simply excluded rather than forcibly repaired. — For Mova (learn): Sentences where witness parsers disagree badly could be left out of the silver data rather than resolved by a guess, keeping the training signal clean.
  • I3. Recording follows a gated loop in which an item enters the database only after passing an automatic check, and otherwise is produced again until it is accepted. — For Mova (learn): LLM teacher outputs could pass through automatic validators (well-formed UD tree, consistent frames) and be re-requested on failure, so only approved annotations reach training.
  • I4. Prompt material is chosen to be balanced, covering every phoneme in each position within a word rather than relying on natural text frequencies. — For Mova (tokens): Morphology and lemma test lists could be built so that every suffix, irregular form and spelling variant appears in each relevant position, exposing gaps that frequency-based samples hide.
  • I5. Results are reported as a ladder of incremental refinements (sample sorting, BatchNorm, early stopping, LM decoding), each with its own error rate. — For Mova (learn): Evaluations could log the score change contributed by each added component, so every improvement is attributable and explainable rather than a single final number.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

3 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1109/ijcnn48605.2020.9206658 · arXiv 2112.14678

@misc{ali2021multi,
  title = {{Multi-Dialect Arabic Speech Recognition}},
  author = {Ali, Abbas Raza},
  year = {2021},
  doi = {10.1109/ijcnn48605.2020.9206658},
  eprint = {2112.14678},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1109/ijcnn48605.2020.9206658}
}

References

28 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Boudraa, B. Boudraa, and B. Guerin, “Twenty Lists of Ten Arabic Sentences for Assessment,” in ACUSTICA, ACTA-ACUSTICA, vol. 86, no. 5, 1998. find
  2. S. Selouani, and J. Caelen, “Arabic phonetic features recognition using modular connectionist architectures,” in Proceedings of the IEEE Interactive Voice Technology for Communication, IVTTA, 1998. find
  3. B. Al-Diri, A. Sharieh, and T. Hudaib, “A time delay neural network architecture for efficient modeling of long temporal contexts,” in Interspeech, 2015. find
  4. V. Peddinti, D. Povey, and S. Khudanpur, “A time delay neural network architecture for efficient modeling of long temporal contexts,” in Interspeech, 2015. find
  5. M. A. Mansour, “KACST arabic phonetics database,” Riyadh, Kingdom of Saudi Arabia, 2004. find
  6. A. Sankaranarayanan, S. Bangalore, and S. S. Narayanan, “Automatic diacritization of Arabic transcripts for automatic speech recognition,” in Proceedings of the International Conference on Natural Language Processing (ICON), 2005. find
  7. M. Alghamdi, F. Alhargan, M. Alkanhal, A. Alkhairy, M. Eldesouki, and A. Alenazi, “Saudi Accented Arabic Voice Bank,” in Journal of King Saud University, Computer and Information Sciences, vol. 20, pages 45–64, 2008. find
  8. X. Glorot, and Y. Bengio, “Understanding the difficulty of training deep feed-forward neural networks,” in Proceedings of the 20 th International Conference on Artificial Intelligence and Statistics (PMLR), 2010. find
  9. A. R. Ali, and S. Hussain, “Automatic Diacritization for Urdu,” in Proceedings of the Conference on Language and Technology (CLT), 2010. find
  10. M. A. M. Abushariah, R. N. Ainon, R. Zainuddin, M. Elshafei, and O. O. Khalifa, “Phonetically rich and balanced text and speech corpora for Arabic language,” in Journal Language Resources and Evaluation, vol. 46, no. 4, pages 601–634, 2012. find
  11. K. Heafield, I. Pouzyrevsky, J. H. Clark, and P. Koehn, “Scalable modified Kneser-Ney language model estimation,” in Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 2013. find our card
  12. A. Graves, A. Mohamed, and G. Hinton, “Speech Recognition with Deep Recurrent Neural Networks,” in Computing Research Repository (CoRR), vol. abs/1303.5778, 2013. ↗
  13. M. Alsulaiman, Z. Ali, G. Muhammed, M. Bencherif, and A. Mahmood, “KSU Speech Database: Text Selection, Recording and Verification,” in European Modelling Symposium, 2013. find
  14. A. Hannun, C. Case, J. Casper, and B. Catanzaro, “Deep Speech: Scaling up end-to-end speech recognition,” in Computing Research Repository (CoRR), vol. abs/1412.5567, 2014. ↗ our card
  15. A. Graves, and N. Jaitly, “Towards end-to-end speech recognition with recurrent neural networks,” in Proceedings of the 31st International Conference on International Conference on Machine Learning (ICML), 2014. find
  16. A. Alalshekmubarak, and L. S. Smith, “On Improving the Classification Capability of Reservoir Computing for Arabic Speech Recognition,” in 24 th International Conference on Artificial Neural Networks, 2014. find
  17. D. P. Kingma, and J. B. Adam, “A method for stochastic optimization,” International Conference on Learning Representations (ICLR), 2015. find
  18. D. Amodei, S. Ananthanarayanan, R. Anubhai, and J. Bai, “Deep Speech 2: End-to-End Speech Recognition in English and Mandarin,” in Proceedings of Machine Learning Research (PMLR), vol. 48, 2015. find our card
  19. S. Wray, and A. Ali, “Crowdsource a little to label a lot: Labeling a Speech Corpus of Dialectal Arabic,” in Interspeech, 2015. find
  20. S. Khurana, and A. Ali, “QCRI advanced transcription system (QATS) for the Arabic Multi-Dialect Broadcast media recognition: MGB-2 challenge,” in IEEE Spoken Language Technology Workshop (SLT), 2016. find
  21. A. Ahmed, Y. Hifny, K. Shaalan, and S. L. Toral, “Lexicon Free Arabic Speech Recognition Recipe,” in International Conference on Advanced Intelligent Systems and Informatics, vol. 12, pages 147–159, 2016. find
  22. T. AlHanai, W. N. Hsu, and J. Glass, “Development of the MIT ASR system for the 2016 Arabic Multi-genre Broadcast Challenge,” in IEEE Spoken Language Technology Workshop (SLT), 2016. find
  23. T. Zerrouki, and A. Balla, “Tashkeela: Novel corpus of Arabic vocalized texts, data for auto-diacritization systems,” in Data in Brief, vol. 11, 2017. find
  24. M. Najafian, W. N. Hsu, A. Ali, and J. Glass, “Automatic speech recognition of Arabic multi-genre broadcast media,” in IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2017. find
  25. A. Ali, P. Bell, J. Glass, Y. Messaoui, H. Mubarak, S. Renals, and Y. Zhang, “The MGB-2 Challenge: Arabic Multi-Dialect Broadcast Media Recognition,” in Computing Research Repository (CoRR), vol. abs/1609.05625, 2017. ↗ our card
  26. M. A. Menacer, O. Mella, and D. Fohr, “An enhanced automatic speech recognition system for Arabic,” in the 3 rd Arabic Natural Language Processing Workshop (EACL), 2017. find
  27. L. Bouchakour, and M. Debyeche, “Improving Continuous Arabic Speech Recognition over Mobile Networks DSR and NSR Using MFCCs Features Transformed,” in International Journal of Circuits, Systems and Signal Processing, vol. 12, 2018. find
  28. A. R. Ali, “Cognitive Computing to Optimize IT Services,” in IEEE 17 th International Conference on Cognitive Informatics and Cognitive Computing (ICCI*CC), Berkeley, CA, 2018, vol. 1, pages 54–60. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ali-2021-multi-dialect-arabic-speech-recognition/