Elamur Research⌕
☰
home / papers / 2023 / Aswat: Arabic Audio Dataset for Automatic Speech Recognition Using Speech-Representation Learning

Aswat: Arabic Audio Dataset for Automatic Speech Recognition Using Speech-Representation Learning

Paper · 2023 · Lamya Alkanhal, Abeer Alessa, Elaf Almahmoud et al.

Summary

Arabic speech recognition has little labeled audio, and the language combines many dialects with orthographic variation. The authors address this by releasing Aswat, a manually cleaned collection of 732 hours of unlabeled Arabic speech. It was gathered from YouTube and SoundCloud and covers audiobooks, news, podcasts and lectures across many topics. Most of it is MSA, with a sizeable Saudi share. They pretrained the self-supervised frameworks wav2vec 2.0 and data2vec (BASE configurations in fairseq) on three sources: Aswat alone, MGB-2 audio alone, and both combined. They then fine-tuned with CTC and evaluated on Common Voice and MGB-2, with and without an n-gram language model. The best system was data2vec pretrained on the combined data and decoded with an LM. It reached WERs of 11.7% on CV and 10.3% on MGB-2, below the previously reported 12.1%. The work gives Arabic ASR both a reusable pretraining resource and reference baselines for two major self-supervised models. It also shows that careless text normalization can hide gains.

Key points

  • Aswat consists of 1060 source files cut into segments of 3–27 s at silences with Pydub, converted to 16 kHz mono and cleaned by hand in Audacity. The split is 724.6 h train and 7.3 h validation, released for non-commercial use.
  • Dialect mix is roughly 66% MSA, 27% Saudi and 7% other dialects. Speech type is about 45% monologue and 55% dialogue, with no overlapping speech.
  • Benchmarks were Common Voice 8.0 (Arabic) and MGB-2. MGB-2's 130M-word Al-Jazeera text corpus served for language modeling, with beam sizes 5 and 20.
  • data2vec outperformed wav2vec in every setting. Adding Aswat to MGB-2 pretraining gave the lowest WER for both frameworks.
  • Converting numbers to words with PyArabic forced MSA nominative forms and caused substitution errors on dialectal numerals. Removing training items that contained numbers fixed this and lowered MGB-2 WER from 11.4% to 10.3%.

Interesting ideas

  • I1. A normalizer that maps every number to one canonical form introduced systematic, measurable errors, and simply dropping the affected training items worked better than keeping them mis-normalized.
  • I2. Errors were sorted into concrete linguistic buckets (numerals, Hamza variants, tāʾ marbūṭah vs hāʾ, dropped clitics like the article and attached conjunctions), each traced to a cause in preprocessing or data.
  • I3. Stripping Latin letters from the training transcripts taught the model to silently delete words spoken with an English accent, so filtering targets can turn into learned omissions.
  • I4. The ablation kept hyperparameters fixed and varied only the pretraining source (general, in-domain, combined), which isolates what each data source contributes.
  • I5. In-domain unlabeled data that overlaps the target task helped most, and mixing it with broader data helped further.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when converting treebanks or building silver data, Mova could discard sentences where a converter is known to be unreliable rather than train on forced, possibly wrong outputs.
  • I2 (tokens): Mova's tokenizer and tagger evaluation could report errors by category (contractions, possessive 's, hyphenated compounds) so fixes target a specific rule instead of an overall score.
  • I3 (tokens): foreign words, symbols and odd tokens should stay in tagger training data with an explicit placeholder tag, so Mova does not learn to skip or mangle them.
  • I4 (learn): Mova could measure the value of each silver or teacher data source by retraining the parser with only that source swapped in, as a clean data-attribution test.
  • I5 (learn): parser or reader training for fables and school tests could combine generic silver text with unlabeled in-genre stories rather than relying on either alone.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Massachusetts Institute of TechnologyCambridge · United States211 works · 69,000 citations🎓King Saud UniversityRiyadh · Saudi Arabia37 works · 527 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksSpeech & spoken language · 480 works

Cite

Cite as Lamya Alkanhal, Abeer Alessa, Elaf Almahmoud and Rana Alaqil. 2023. Aswat: Arabic Audio Dataset for Automatic Speech Recognition Using Speech-Representation Learning. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.10

@inproceedings{alkanhal2023aswat,
  title = {{Aswat: Arabic Audio Dataset for Automatic Speech Recognition Using Speech-Representation Learning}},
  author = {Lamya Alkanhal, Abeer Alessa, Elaf Almahmoud and Rana Alaqil},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.10},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.10}
}

References

19 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelrahman, Yasser Hifny, Khaled Shaalan, and Sergio Toral. 2018. “End-to-End Lexicon Free Arabic Speech Recognition Using Recurrent Neural Networks.” Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui Hamdy Mubarak, Steve Renals, and Yifan Zhang. 2016. “The MGB-2 challenge: Arabic multi-dialect broadcast media recognition”. In Proc IEEE SLT. ↗
  2. Ahmed Ali, Stephan Vogel, and Steve Renals. 2017. “Speech recognition challenge in the wild: Arabic MGB-3. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU).
  3. Ahmed Ali, Suwon Shon, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, James Glass, Steve Renals, and Khalid Choukri. 2019. “The MGB-5 Challenge: Recognition and Dialect Identification of Dialectal Arabic Speech”. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) find
  4. Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. “Wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations.” Advances in Neural Information Processing Systems 2020-December: 1–12.
  5. Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. 2022. “data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language,”. In Proceedings of the 39th International Conference on Machine Learning. PMLR. ↗
  6. Alex Graves, and Navdeep Jaitly. 2014. “Towards End-to-End Speech Recognition with Recurrent Neural Networks.” 31st International Conference on Machine Learning, ICML. find
  7. Amir Hussein, Shinji Watanabe, and Ahmed Ali. 2022. “Arabic Speech Recognition by End-to-End, Modular Systems and Human.” Computer Speech and Language 71: 1–39. ↗
  8. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. “Attention is all you need.” In 9 https://github.com/wiseman/py-webrtcvad Advances in Neural Information Processing Systems, pages 6000–6010. ↗
  9. Recognition and Understanding Workshop, ASRU 2019 - Proceedings 9(4): 449–56. find
  10. David Doukhan, Eliott Lechapt, Marc Evrard, and Jean Carrive. 2018. “Ina’s mirex 2018 music and speech detection system.” Music Information Retrieval Evaluation eXchange (MIREX 2018). find
  11. Suwon Shon, Ahmed Ali, Younes Samih, Hamdy Mubarak, and James Glass. 2020. “ADI17: A Fine-Grained Arabic Dialect Identification Dataset”. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) find
  12. Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. 2021. “QASR: QCRI Aljazeera Speech Resource A Large Scale Annotated Arabic Speech Corpus.” In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2274–2285, Online. Association for Computational Linguistics. ↗
  13. Karin C. Ryding. 2005. “A Reference Grammar of Modern Standard Arabic”. Cambridge: Cambridge University Press, 2005. ↗
  14. Mohammad Al-Fetyani, Muhammad Al-Barham, Gheith Abandah, Adham Alsharkawi, and Maha Dawas. 2021. “MASC: Massive Arabic Speech Corpus.” IEEE Spoken Language Technology Workshop (SLT). doi:10.21227/e1qb-jv46 ↗
  15. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. “HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units.” IEEE/ACM Transactions on Audio Speech and Language Processing 29(Cv): 3451–60. ↗
  16. Yonatan Belinkov, Ahmed Ali, and James Glass. 2019. “Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognition.” Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. find
  17. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. “FAIRSEQ: A fast, extensible toolkit for sequence modeling.” In North American Association for Computational Linguistics (NAACL): System Demonstrations. ↗
  18. Shammur Absar Chowdhury, Amir Hussein, Ahmed Abdelali, and Ahmed Ali. 2021. “Towards one model to rule all: Multilingual strategy for dialectal codeswitching Arabic.” Interspeech 2021. pages. 2466–2470. [Online]. Available: https://www.isca-speech. ↗
  19. Shigeki Karita, Nanxin Chen, Tomoki Hayashi, Takaaki Hori, Hirofumi Inaguma, Ziyan Jiang, Masao Someki, Nelson Enrique Yalta Soplin, Ryuichi Yamamoto, Xiaofei Wang, Shinji Watanabe, Takenori Yoshimura, Wangyou Zhang. 2019. “A Comparative Study on Transformer vs RNN in Speech Applications.” 2019 IEEE Automatic Speech find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alkanhal-2023-aswat-arabic-audio-dataset-automatic/