Elamur Research⌕
☰
home / papers / 2025 / Advancing African-Accented English Speech Recognition: Epistemic Uncertainty-Driven Data Selection for Generalizable ASR Models

Advancing African-Accented English Speech Recognition: Epistemic Uncertainty-Driven Data Selection for Generalizable ASR Models

Paper · 2025 · Bonaventure F. P. Dossou

Summary

Commercial and open ASR systems handle African-accented English poorly, because such accents are rare in their training data and transcribed speech is costly to collect. Dossou adapts pretrained speech models to these accents with a multi-round active-learning loop that uses epistemic uncertainty (EU) to decide which recordings the model should learn from next. The loop combines a core-set-style small starting subset, Deep Bayesian Active Learning with MC-Dropout to estimate EU, and an adversarial-flavoured uncertainty-sampling acquisition step. Each round fine-tunes on the current training set, scores the remaining pool, and moves the top-k most uncertain samples into training. A new metric, U-WER, follows how the model's variance on difficult accents changes over the rounds.

Experiments span AfriSpeech-200 and three external corpora, the general, clinical and combined domains, and four speech models. Relative to baselines, WER improves by about 27% on average while using roughly 45% less labelled data, and very low-resource accents also benefit out of distribution. This makes accent adaptation cheaper where annotation budgets are tight.

Key points

  • Main corpus. AfriSpeech-200 has 200.91 hours, 67,577 clips, 2,463 speakers and more than 120 accents from 13 anglophone African countries plus the US. About 62% of the clips are clinical and 38% general.
  • External datasets. These are SautiDB (919 Nigerian-accented clips, 59 minutes), MedicalSpeech (6,661 utterances, 8 hours) and the CommonVoices English Accented Dataset (v10, with western accents removed).
  • Models. The method is tested on Wav2Vec2-XLSR-53, HuBERT-Large, WavLM-Large and NVIDIA Conformer-CTC Large. The EU-Most strategy, found best in the Wav2Vec runs, was reused for all the other models.
  • Setup. Data starts split into 30% training and 70% pool. MC-Dropout uses T=10 stochastic forward passes per sample.
  • Results. Relative average WER improvement is 27.00% (general), 15.51% (clinical) and 26.56% (both). The approach also beats Whisper-Medium.
  • Limitation. The clinical domain needs a stopping criterion to balance the number of rounds against pool size.
  • Future work. Deep Ensembles, BALD and BatchBALD are left as alternatives for when more compute is available.

Interesting ideas

  • I1. Running the same model several times with dropout active at inference turns disagreement between its outputs into a cheap estimate of what it does not yet know, without training an ensemble.
  • I2. When gold transcripts are absent, one of the T stochastic outputs is picked as the pseudo-label through WER-based comparison among them, so the model labels its own pool.
  • I3. U-WER does not just track average error; it monitors how prediction variance on the hardest subgroups shrinks across adaptation rounds, showing whether the model is actually settling on difficult cases.
  • I4. Training starts from a deliberately small seed set and grows it only with the most informative samples, reaching better scores with far less labelled material than training on everything.
  • I5. Results are broken down per accent, including out-of-distribution, very low-resource ones, rather than reported only as a global average.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Sentences on which gparse's dropout passes produce conflicting trees could be routed first to the LLM teacher tiers or to human checking, spending annotation budget where the parser is least sure.
  • I2 (learn): Dropout passes of a single parser could serve as extra "witnesses" alongside the existing witness parsers, with the most mutually consistent tree kept as silver data.
  • I3 (tree): An analogous variance-of-LAS measure could be tracked per treebank or UD dialect across training rounds, flagging conversions where the parser is still unstable.
  • I4 (learn): Treebank or silver-data training could begin from a compact core and add uncertainty-selected sentences per round, cutting compute and exposing which constructions actually drive gains.
  • I5 (answer): Reader and student evaluations could be reported separately for each text type (fables, maths word problems, school tests), with rare genres treated as OOD probes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓McGill UniversityMontreal · Canada87 works · 11,027 citations🔬Mila - Quebec Artificial Intelligence InstituteMontreal · Canada54 works · 13,565 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksSpeech & spoken language · 480 works

Cite

Cite as Bonaventure F. P. Dossou. 2025. Advancing African-Accented English Speech Recognition: Epistemic Uncertainty-Driven Data Selection for Generalizable ASR Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-srw.1

@inproceedings{dossou2025advancing,
  title = {{Advancing African-Accented English Speech Recognition: Epistemic Uncertainty-Driven Data Selection for Generalizable ASR Models}},
  author = {Bonaventure F. P. Dossou},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-srw.1},
  url = {https://doi.org/10.18653/v1/2025.acl-srw.1}
}

References

42 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tejumade Afonja, Clinton Mbataku, Ademola Malomo, Olumide Okubadejo, Lawrence Francis, Munachiso Nwadike, and Iroro Orife. 2021a. Sautidb: Nigerian accent dataset collection. find
  2. Alex DiChristofano, Henry Shuster, Shefali Chandra, and Neal Patwari. 2022. Performance disparities between accents in automatic speech recognition. arXiv preprint arXiv:2208.01157. ↗
  3. Tejumade Afonja, Oladimeji Mudele, Iroro Orife, Kenechi Dukor, Lawrence Francis, Duru Goodness, Oluwafemi Azeez, Ademola Malomo, and Clinton Mbataku. 2021b. Learning nigerian accent embeddings from speech: preliminary results based on sautidb-naija corpus. arXiv preprint arXiv:2112.06199. ↗
  4. Bonaventure FP Dossou and Chris C Emezue. 2021. Okwugb\’e: End-to-end speech recognition for fon and igbo. arXiv preprint arXiv:2103.07762. ↗
  5. Bonaventure FP Dossou, Atnafu Lambebo Tonja, Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, and Chris Chinenye Emezue. 2022. Afrolm: A self-active learning-based multilingual pretrained language model for 23 african languages. arXiv preprint arXiv:2211.03263. ↗
  6. Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. 2019. Common voice: A massively-multilingual speech corpus. arXiv preprint arXiv:1912.06670. ↗
  7. Melanie Ducoffe and Frederic Precioso. 2018. Adversarial active learning for deep networks: a margin based approach. arXiv preprint arXiv:1802.09841. ↗
  8. Jaco Badenhorst and Febe De Wet. 2017. The limitations of data perturbation for asr of learner data in under-resourced languages. In 2017 Pattern Recognition Association of South Africa and Robotics and Mechatronics (PRASA-RobMech), pages 44–49. IEEE. find
  9. David Eberhard, Gary Simons, and Chuck Fennig. 2019. Ethnologue: Languages of the World, 22nd Edition. find
  10. Gregory Finley, Wael Salloum, Najmeh Sadoughi, Erik Edwards, Amanda Robinson, Nico Axtmann, Michael Brenndoerfer, Mark Miller, and David Suendermann-Oeft. 2018. From dictations to clinical reports using machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers), pages 121–128. find
  11. Jaco Badenhorst and Febe De Wet. 2019. The usefulness of imperfect speech data for asr development in low-resource languages. Information, 10(9):268. ↗
  12. Etienne Barnard, Marelie Davel, and Charl Van Heerden. 2009. Asr corpus design for resource-scarce languages. ISCA. ↗
  13. M. Benzeghiba, R. De Mori, O. Deroo, S. Dupont, T. Erbes, D. Jouvet, L. Fissore, P. Laface, A. Mertins, C. Ris, R. Rose, V. Tyagi, and C. Wellekens. 2007. Automatic speech recognition and speech variability: A review. Speech Communication, 49(10):763–786. Intrinsic Speech Variations. ↗
  14. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059. PMLR. ↗
  15. Yarin Gal, Riashat Islam, and Zoubin Ghahramani. 2017. Deep Bayesian active learning with image data. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. 2022. of Machine Learning Research, pages 1183–1192. PMLR. Advances in neural information processing systems, 30. ↗
  16. Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al. 2020. Conformer: Convolution-augmented transformer for speech recognition. arXiv preprint arXiv:2005.08100. ↗
  17. Shang Liu and Xiaocheng Li. 2023. Understanding uncertainty sampling. arXiv preprint arXiv:2307.02719. ↗
  18. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM Comput. Surv., 54(6). ↗
  19. Dilek Hakkani-Tür, Giuseppe Riccardi, and Allen Gorin. 2002. Active learning for automatic speech recognition. In 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 4, pages IV–3904–IV–3907. find
  20. Zion Mengesha, Courtney Heldreth, Michal Lahav, Juliana Sublewski, and Elyse Tuennerman. 2021. “i don’t think these devices are very culturally sensitive.”—impact of automated speech recognition errors on african americans. Frontiers in Artificial Intelligence, page 169. ↗
  21. Arthur Hinsvark, Natalie Delworth, Miguel Del Rio, Quinten McNamara, Joshua Dong, Ryan Westerman, Michelle Huang, Joseph Palakapilly, Jennifer Drexler, Ilya Pirkin, Nishchal Bhandari, and Miguel Jette. 2021. Accented speech recognition: A survey. ↗
  22. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, page 220–229, New York, NY, USA. Association for Computing Machinery. find
  23. Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. 2011. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:1112.5745. ↗
  24. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3451–3460. ↗
  25. Udhyakumar Nallasamy, Florian Metze, and Tanja Schultz. 2012. Active learning for accent adaptation in automatic speech recognition. In 2012 IEEE Spoken Language Technology Workshop (SLT), pages 360–365. find
  26. Alex Kendall and Yarin Gal. 2017. What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neural Information Processing Systems, 30:5574–5584. ↗
  27. Tobi Olatunji, Tejumade Afonja, Bonaventure F. P. Dossou, Atnafu Lambebo Tonja, Chris Chinenye Emezue, Amina Mardiyyah Rufai, and Sahib Singh. 2023a. AfriNames: Most ASR Models "Butcher" African Names. In Proc. INTERSPEECH 2023, pages 5077– 5081. ↗
  28. Andreas Kirsch, Joost van Amersfoort, and Yarin Gal. 2019. BatchBALD: efficient and diverse batch acquisition for deep Bayesian active learning. Curran Associates Inc., Red Hook, NY, USA. ↗
  29. Tobi Olatunji, Tejumade Afonja, Aditya Yadavalli, Chris Chinenye Emezue, Sahib Singh, Bonaventure F. P. Dossou, Joanne Osuchukwu, Salomey Osei, Atnafu Lambebo Tonja, Naome A. Etori, and Clinton Mbataku. 2023b. Afrispeech-200: Pan-african accented speech dataset for clinical and general domain asr. ↗
  30. Jodi Kodish-Wachs, Emin Agassi, Patrick Kenny III, and J Marc Overhage. 2018. A systematic comparison of contemporary automatic speech recognition engines for conversational clinical speech. In AMIA Annual Symposium Proceedings, volume 2018, page 683. American Medical Informatics Association. find
  31. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. ↗
  32. Allison Koenecke. 2021. Racial Disparities in Automated Speech Recognition. In Proc. 2021 ISCA Symposium on Security and Privacy in Speech Communication. ↗
  33. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492–28518. PMLR. ↗
  34. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684–7689. ↗
  35. G. Riccardi and D. Hakkani-Tur. 2005. Active learning: theory and applications to automatic speech recognition. IEEE Transactions on Speech and Audio Processing, 13(4):504–511. ↗
  36. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. A.5 ↗
  37. Andrew D. Selbst, Danah Boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. Fairness and abstraction in sociotechnical systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, page 59–68, New York, NY, USA. Association for Computing Machinery. Figure 7 shows ascending and descending accents across the Top 2k most uncertain samples. find
  38. Ozan Sener and Silvio Savarese. 2017. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489. ↗
  39. Burr Settles. 2009. Active learning literature survey.
  40. Yulia Tsvetkov. 2017. Opportunities and challenges in working with low-resource languages. In Carnegie Mellon Univ., Language Technologies Institute. find
  41. Patrice Yemmene and Laurent Besacier. 2019. Motivations, challenges, and perspectives for the development of an automatic speech recognition system for the under-resourced ngiemboon language. In Proceedings of The First International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2019) co-located with ICNLSP 2019-Short Papers, pages 59–67. find
  42. Julián Zapata and Andreas Søeborg Kirkedal. 2015. Assessing the performance of automatic speech recognition systems when used by native and non-native speakers of three major languages in dictation workflows. In Proceedings of the 20th Nordic Conference of Computational Linguistics (NODALIDA 2015), pages 201–210. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dossou-2025-advancing-african-accented-english-speech/