Elamur Research⌕
☰
home / papers / 2024 / Arabic Speech Recognition of zero-resourced Languages: A case of Shehri (Jibbali) Language

Arabic Speech Recognition of zero-resourced Languages: A case of Shehri (Jibbali) Language

Summary

The paper tackles automatic speech recognition for Shehri (Jibbali), a Modern South Arabian language of Dhofar, Oman. The language has almost no annotated speech, no widely fixed writing system, and a shrinking number of young speakers. The authors built a small public speech corpus. Thirty speakers (23 male, 7 female) each read 15 sentences five times, giving about 1 h 54 min of audio. The sentences were chosen to contain Shehri sounds that Arabic lacks. Transcriptions use an Arabic-based alphabet with extra letters for those sounds, agreed after consulting the experts Watson and Al-Kathiri. Three pretrained speech model families were then fine-tuned on an 80/20 random split and scored with WER and CER: Wav2Vec2.0, HuBERT and Whisper. Whisper clearly won, and Whisper-medium reached 3.5% WER. The authors present this as evidence that transfer learning can bootstrap ASR for documenting endangered languages.

One caveat follows from the setup: the corpus has only 15 distinct sentences, and utterances were shuffled before splitting. Test transcripts therefore also appear in training, so the scores likely overstate how well the models generalise.

Key points

  • Corpus: 30 speakers × 15 sentences × 5 repetitions = 75 utterances per speaker, about 2,250 in total.
    • Speakers come from eastern and central Dhofar plus the adjacent western Asʼaib area.
    • Average clip length is about 3 s.
    • The corpus is released on GitHub (iwan-rg/Shehri-Jibbali-Speech-Dataset).
  • Split: random shuffle, then 80% train (about 1 h 31 min) and 20% test (about 22 min).
  • Models compared: Wav2Vec2.0, HuBERT and Whisper, evaluated with WER and CER.
  • Best result: Whisper-medium at 3.5% WER, well ahead of the self-supervised encoders.
  • Writing system: Shehri shares 25 consonant letters with Standard Arabic. It adds letters such as ڛ, ڸ and ڞ for sounds like /ɬ/, /ɮ/, /tʃʼ/ and ejectives, following Al-Kathiri et al. (2024).

Interesting ideas

  • I1. Build the elicitation material around exactly the phenomena the borrowed model has never seen (here, non-Arabic phonemes), so a tiny dataset spends its budget where transfer is weakest.
  • I2. Fix the transcription convention with domain experts before annotating, so every label follows one agreed inventory instead of ad-hoc spellings.
  • I3. Report two granularities of error (word-level and character-level) side by side, which separates near-miss spellings from completely wrong words.
  • I4. Fine-tune several pretrained model families on the same small data and the same split, which turns the choice of base model into a measured decision rather than a default.
  • I5. The design shows a trap: with few distinct sentences, shuffling individual recordings lets test items share text (and speakers) with training, which can inflate scores.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when collecting teacher or silver data, target the constructions the witness parsers handle worst instead of sampling text uniformly.
  • I2 (tree): before converting UD dialects or merging treebanks, freeze one explicit mapping of labels and features so the training signal stays consistent.
  • I3 (tokens): score lemmas and morphology both as whole-token accuracy and at character or feature level, to tell small inflection slips from wrong analyses.
  • I4 (learn): benchmark the perceptron parser and gparse variants on identical small silver sets before choosing which one to adapt.
  • I5 (learn): split evaluation data by source text or document, and add a negative control on unseen sentences to expose this kind of leakage.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics Speech & spoken language Semitic languages

Cite

Cite as Norah A. Alrashoudi, Omar Said Alshahri and Hend Al-Khalifa. 2024. Arabic Speech Recognition of zero-resourced Languages: A case of Shehri (Jibbali) Language. Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alrashoudi2024arabic,
  title = {{Arabic Speech Recognition of zero-resourced Languages: A case of Shehri (Jibbali) Language}},
  author = {Norah A. Alrashoudi, Omar Said Alshahri and Hend Al-Khalifa},
  booktitle = {Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024},
  year = {2024},
  doi = {10.63317/5ozhxig2mmtz},
  url = {https://doi.org/10.63317/5ozhxig2mmtz}
}

References

12 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Accessed: Nov. 06, 2023. [Online]. Available: http://arxiv.org/abs/2106.09685 — (2003). Lisān Ẓufār al-ḥimyarī al-muʿāṣir. Dirāsah G. V. Stan, A. Baart, F. Dittoh, H. Akkermans, and A. muʿjamiyyah muqāranah, Jāmiʿat al-Sulṭān Bon, “A Lightweight Downscaled Approach to Qābūs, Markaz al-dirāsāt al-ʿumāniyyah. Automatic Speech Recognition for Small “Ethnologue: Languages of Africa and Europe, Indigenous Languages,” in 14th ACM Web Twenty-Fifth Edition,” SIL International, 2022. ↗ our card
  2. Science Conference 2022, Barcelona Spain: Accessed: Feb. 18, 2024. [Online]. Available: ACM, Jun. 2022, pp. 451–458. doi: https://www.sil.org/resources/publications/entry 10.1145/3501247.3539017. /93719 G. Zellou and M. Lahrouchi, “Linguistic disparities in A. Babu et al., “XLS-R: Self-supervised Cross-lingual cross-language automatic speech recognition Speech Representation Learning at Scale,”, transfer from Arabic to Tashlhiyt,” Sci. ↗
  3. Rep., 2021 arXiv.org. Accessed: Aug. 08, 2023. vol. 14, no. 1, Art. no. 1, Jan. 2024, doi: [Online]. Available: 10.1038/s41598-023-50516-3. https://arxiv.org/abs/2111.09296v3 ↗ our card
  4. Johnstone. (1981). Jibbali Lexicon. London: Oxford A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, University Press. “wav2vec 2.0: A Framework for Self-Ministry of Heritage and Culture. (2013). The Omani Supervised Learning of Speech Encyclopedia (Vol. 6). Muscat: Ministry of Representations,” in Advances in Neural Heritage and Culture. find
  5. Information Processing Systems, Curran N. Das, S. Bodapati, M. Sunkara, S. Srinivasan, and Associates, Inc., 2020, pp. 12449–12460. D. H. Chau, “Best of Both Worlds: Robust Accessed: Apr. 06, 2022. [Online]. Available: Accented Speech Recognition with Adversarial https://proceedings.neurips.cc/paper/2020/has Transfer Learning.” arXiv, Mar. 09, 2021. h/92d1e1eb1cd6f9fba3227870bb6d7f07-Accessed: Feb. 18, 2024. [Online]. Available: Abstract.html http://arxiv.org/abs/2103.05834 A. Radford, J. W. Kim, T. Xu, G. Brockman, C. National Center for Statistics and Information. McLeavey, and I. Sutskever, “Robust Speech (2020). Census 2020. Muscat: National Center Recognition via Large-Scale Weak for Statistics and Information. Supervision.” arXiv, Dec. 06, 2022. Accessed: P. Arisaputra, A. T. Handoyo, and A. Zahra, “XLS-R Nov. 05, 2023. [Online]. Available: Deep Learning Model for Multilingual ASR on http://arxiv.org/abs/2212.04356 Low-Resource Languages: Indonesian, Āl Ḥāfiẓ, ʿAlī Muḥsin. (1987). Min lahajāt “mahrah” Javanese, and Sundanese.” arXiv, Jan. 12, wa-’ādābihā. Majallat an-nahḍah al-2024. Accessed: Jan. 31, 2024. [Online]. ʿumāniyyah. Muscat. Available: http://arxiv.org/abs/2401.06832 ↗
  6. Al-Kathiri, A, Al-Mashani, A & Alshahri, O. (2024). Al-R. Coto-Solano et al., “Development of Automatic Luban Wal-Turath Al-Thaqafi. Ministry of Speech Recognition for the Documentation of Culture, Sports, and Youth, Literary Forum. Cook Islands Māori”, 2022. Muscat, Sultanate of Oman. find
  7. Rubin, A. D. (2014). The Jibbali (Shahri) language of Al-Kathiri, Amer; Al-Maashani, Abdulaziz; Al-Kathiri, Oman: grammar and texts. In The Jibbali Salem. (In press). Al-Wadu' al-lughawi fi (Shaḥri) Language of Oman. Brill. Dhofar: Dirasat lughawiyat ijtimaiyah. Arab S. Young et al., The HTK Book, vol. 3.4. 1995. Journal of Humanities Sciences. Accessed: Nov. 05, 2023. [Online]. Available: Al-Maashani, Saeed. (2017). Al-Ishtiqagh fi al-https://www.inf.u-Lughah al-Jabaliiyah: Mawazanah bil Lughah szeged.hu/~tothl/speech/htkbook.pdf al-Arabiyyah. Research and Cognitive T. Dettmers, M. Lewis, Y. Belkada, and L. Communication Center. Riyadh. Zettlemoyer, “LLM.int8(): 8-bit Matrix B. Neyshabur, H. Sedghi, and C. Zhang, “What is Multiplication for Transformers at Scale.” arXiv, being transferred in transfer learning?,” in Nov. 10, 2022. doi: Advances in Neural Information Processing 10.48550/arXiv.2208.07339. ↗
  8. Systems, Curran Associates, Inc., 2020, pp. V. Gupta and G. Boulianne, “Speech Transcription 512–523. Accessed: Feb. 18, 2024. [Online]. Challenges for Resource Constrained Available: Indigenous Language Cree,” in Proceedings of https://proceedings.neurips.cc/paper/2020/has the 1st Joint Workshop on Spoken Language h/0607f4c705595b911a4f3e7a127b44e0-Technologies for Under-resourced languages Abstract.html (SLTU) and Collaboration and Computing for C.-C. Chen, W. Chen, R. Zevallos, and J. E. Ortega, Under-Resourced Languages (CCURL), D. “Evaluating Self-Supervised Speech Beermann, L. Besacier, S. Sakti, and C. Soria, Representations for Indigenous American Eds., Marseille, France: European Language Languages.” arXiv, Oct. 08, 2023. Accessed: Resources association, May 2020, pp. 362– Jan. 22, 2024. [Online]. Available: 367. Accessed: Jan. 22, 2024. [Online]. http://arxiv.org/abs/2310.03639 Available: https://aclanthology.org/2020.sltu-E. J. Hu et al., “LoRA: Low-Rank Adaptation of 1.51 Large Language Models.” arXiv, Oct. 16, 2021. 91 ↗
  9. W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units,” IEEEACM Trans. Audio Speech Lang. Process., vol. 29, pp. 3451–3460, 2021, doi: 10.1109/TASLP.2021.3122291. ↗ our card
  10. Watson, J. C., & Al-Kathiri, A. A. A. (2022). A phonetically “unnatural” class in Central and Eastern Shehret (Jibbali). Kervan: International Journal of Afro-Asiatic Studies, 26(1), 129-159. find
  11. Y. Woldemariam, “Transfer Learning for Less-Resourced Semitic Languages Speech Recognition: the Case of Amharic,” in Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), D. Beermann, L. Besacier, S. Sakti, and C. Soria, Eds., Marseille, France: European Language Resources association, May 2020, pp. 61–69. Accessed: Mar. 30, 2024. [Online]. Available: https://aclanthology.org/2020.sltu-1.9 ↗ our card
  12. Z. Fu, H. Yang, A. M.-C. So, W. Lam, L. Bing, and N. Collier, “On the Effectiveness of Parameter-Efficient Fine-Tuning,” Proc. AAAI Conf. Artif. Intell., vol. 37, no. 11, Art. no. 11, Jun. 2023, doi: 10.1609/aaai.v37i11.26505. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alrashoudi-2024-arabic-speech-recognition-zero-resourced/