Elamur Research⌕
☰
home / papers / 2026 / Doing More with Less: Data Augmentation for Sudanese Dialect Automatic Speech Recognition

Doing More with Less: Data Augmentation for Sudanese Dialect Automatic Speech Recognition

Summary

Sudanese Arabic has very little transcribed speech, and general-purpose recognizers handle it poorly. The paper sets a first Whisper benchmark for this dialect and tests whether cheap data augmentation can stand in for scarce labels. Whisper Small, Medium and Large-V2 were fine-tuned on about 4 hours of Sudanese speech mixed with an MSA studio corpus. Two ways of adding data were then tried on Small and Medium:

  • a teacher model labelled unlabelled broadcast clips, and only labels above a confidence threshold were kept;
  • a new Lisan-Sudanese TTS set was built by synthesizing heavily diacritized dialect sentences with the Klaam system.

The strongest system was Whisper-Medium trained on 28.4 hours of real, pseudo-labelled and synthetic audio. It reached 57.1% WER on OOOK-Eval and 51.6% on an out-of-domain holdout. For comparison, zero-shot Whisper scored 78.8% at best, and Arabic models tuned on MSA scored 73.8–123%.

All training ran on free-tier cloud GPUs, and the models and pipelines are public. This makes the work a practical recipe for other dialects with few resources.

Key points

  • Data:
    • Sudanese Dialect Speech Dataset: 3,544 clips, about 4 h, 31 speakers.
    • Arabic Speech Corpus: MSA studio recordings.
    • OOOK-Eval: about 2 h of broadcast speech drawn from ADI17.
    • Unlabelled OOOK clips: 17,973 were pseudo-labelled.
    • Lisan-Sudanese TTS: 1,878 sentences, 4.6 h of synthetic audio.
  • Plain fine-tuning (7.25 h of MSA+SDN): WER on OOOK-Eval was 67.7 (Small), 64.1 (Medium) and 62.8 (Large-V2). On the holdout, Medium was best at 57.5, ahead of Large-V2. The larger model gains little when data is this limited.
  • Pseudo-label filtering: confidence thresholds of 0.7 and 0.9 were compared. The teacher was trained on a cleaned gold subset only.
  • MSA-tuned models: the ARBML models looked fairly good on OOOK-Eval (73.8–75.5 WER for Medium/Large) but collapsed on the holdout (140–146 WER). The Common Voice model was the worst overall (123 WER).
  • Compute: Kaggle P100 and Lightning.ai L40S free tiers, 5,000 steps, LR 1e-5. The best result is about a 45% relative gain over zero-shot on out-of-domain data.

Interesting ideas

  • I1. Confidence-gated pseudo-labels at two cut-offs, compared side by side, show that the threshold is a lever trading label accuracy against coverage and generalization.
  • I2. The teacher is trained only on a cleaned gold subset before it labels anything, so its mistakes do not compound through the student.
  • I3. Sentences for synthesis are chosen by an annotation-density score (the share of diacritics), keeping only the most fully marked text.
  • I4. A second holdout from a different source exposed models whose in-domain scores hid poor generalization.
  • I5. Real-but-noisy data (pseudo-labels) and clean-but-artificial data (TTS) helped in complementary ways, and the two worked best combined.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Sweep the agreement threshold for silver parses from the witness parsers and report the effect on both in-domain and held-out scores, rather than fixing one cut-off.
  • I2 (learn): Train the LLM-teacher-tier models that produce silver data on curated gold treebank data only, and keep silver out of their training loop.
  • I3 (tokens): Rank candidate training sentences by how densely their morphology and lemmas are annotated, and prefer the richest ones when training the tagger and lemmatizer.
  • I4 (answer): Test each reader on an extra set of fables or school texts from an unseen source, so that overfitting to one test style shows up as a control failure.
  • I5 (tree): Mix silver parses of real text with sentences generated from known meaning graphs, whose trees are exact by construction, when training gparse.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Speech & spoken language Semitic languages

Cite

Cite as Ayman Mansour. 2026. Doing More with Less: Data Augmentation for Sudanese Dialect Automatic Speech Recognition. arXiv:2601.06802.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2601.06802 · arXiv 2601.06802

@misc{mansour2026doing,
  title = {{Doing More with Less: Data Augmentation for Sudanese Dialect Automatic Speech Recognition}},
  author = {Ayman Mansour},
  year = {2026},
  doi = {10.48550/arxiv.2601.06802},
  eprint = {2601.06802},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2601.06802}
}

References

18 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Conneau, A. Baevski, R. Collobert, A. Mohamed, and M. Auli, “Unsupervised cross-lingual representation learning for speech recognition,” arXiv preprint arXiv:2006.13979, 2020. ↗
  2. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International conference on machine learning. PMLR, 2023, pp. 28 492–28 518. our card
  3. Q. Xu, A. Baevski, T. Likhomanenko, P. Tomasello, A. Conneau, R. Collobert, G. Synnaeve, and M. Auli, “Self-training and pre-training are complementary for speech recognition,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 3030–3034. find
  4. A. Mansour and W. F. Mukhtar, “End-to-end automatic speech recognition model for the sudanese dialect,” arXiv preprint arXiv:2212.10826, 2022. ↗
  5. J. Kahn, A. Lee, and A. Hannun, “Self-training for end-to-end speech recognition,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 7084–7088. find
  6. A. Rosenberg, Y. Zhang, B. Ramabhadran, Y. Jia, P. Moreno, Y. Wu, and Z. Wu, “Speech recognition with augmented synthesized speech,” in 2019 IEEE automatic speech recognition and understanding workshop (ASRU). IEEE, 2019, pp. 996–1002. find
  7. M. Bartelds, N. San, B. McDonnell, D. Jurafsky, and M. Wieling, “Making more of little data: Improving low-resource automatic speech recognition using data augmentation,” arXiv preprint arXiv:2305.10951, 2023. ↗
  8. B. Talafha, A. Waheed, and M. Abdul-Mageed, “N-shot benchmarking of whisper on diverse arabic speech recognition,” arXiv preprint arXiv:2306.02902, 2023. ↗ our card
  9. Ö. T. Özyilmaz, M. Coler, and M. Valdenegro-Toro, “Overcoming data scarcity in multi-dialectal arabic asr via whisper fine-tuning,” arXiv preprint arXiv:2506.02627, 2025. ↗ our card
  10. N. Halabi and M. Wald, “Phonetic inventory for an arabic speech corpus,” 2016. find
  11. M. Jarrar, F. A. Zaraket, T. Hammouda, D. M. Alavi, and M. Wählisch, “Lisan: Yemeni, iraqi, libyan, and sudanese arabic dialect corpora with morphological annotations,” in 2023 20th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA). IEEE, 2023, pp. 1–7. find
  12. Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y. Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” arXiv preprint arXiv:2006.04558, 2020. ↗
  13. S. Shon, A. Ali, Y. Samih, H. Mubarak, and J. Glass, “Adi17: A fine-grained arabic dialect identification dataset,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 8244–8248. our card
  14. R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in Proceedings of the twelfth language resources and evaluation conference, 2020, pp. 4218–4222. our card
  15. A. Ali, P. Bell, J. Glass, Y. Messaoui, H. Mubarak, S. Renals, and Y. Zhang, “The mgb-2 challenge: Arabic multi-dialect broadcast media recognition,” in 2016 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2016, pp. 279–284. our card
  16. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al., “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38–45. our card
  17. N. Y. Habash, Introduction to Arabic natural language processing. Morgan & Claypool Publishers, 2010. find
  18. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models.” ICLR, vol. 1, no. 2, p. 3, 2022. our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mansour-2026-doing-more-less-data-augmentation/