Elamur Research⌕
☰
home / papers / 2025 / Automatic Speech Recognition of African American English: Lexical and Contextual Effects

Automatic Speech Recognition of African American English: Lexical and Contextual Effects

Paper · 2025 · Hamid Mojarad and Kevin Tang

Summary

The paper asks why end-to-end speech recognizers make more errors on African American English (AAE), focusing on two pronunciation features. The first is consonant cluster reduction (CCR), where a final stop is dropped, as in "cold" said as "col'". The second is ING-reduction, where -ing is said as -in. The authors used the Washington DC subcorpus (DCA) of CORAAL and transcribed it with wav2vec 2.0, both alone and with an external 5-gram KenLM language model. They found which word tokens were actually reduced by force-aligning the audio with the Montreal Forced Aligner, using a dictionary that listed both full and reduced pronunciations. A misrecognition was counted as a lexical-neighborhood error when the ASR output was one phoneme edit away from the target's realized form. Linear mixed-effects models show that reduced pronunciations raise WER by a small but significant amount. Neighborhood confusions are more common when no LM is used. This suggests that context modelling partly compensates for acoustic ambiguity caused by dialect features. That matters for diagnosing and reducing dialect bias in ASR.

Key points

  • Data: CORAAL DCA subcorpus, with 74 recordings from 68 speakers (40 men, 28 women) across four age groups and three socioeconomic classes. It totals about 34 hours and 333,500 words. Only interviewee speech was analysed.
  • ASR setup: facebook/wav2vec2-large-960h, run with and without a 5-gram KenLM model. The LM was trained on CORAAL DCA+DCB, so it is in-domain for the DC data.
    • Audio was resampled to 16 kHz.
    • Audio was cut into chunks of at least 30 s without splitting utterances.
    • Output was aligned to references with Needleman-Wunsch from the string2string library.
  • Feature detection: MFA 2.2.17 with the CMU Pronouncing Dictionary. A g2p model filled in missing words. Reduced variants were added for CCR- and ING-prone words, and a custom acoustic model was trained on DCA.
  • Neighbor definition follows Luce: one phoneme substitution, deletion or insertion, computed with Levenshtein distance. Neighbors were computed from the reduced pronunciation whenever MFA detected the reduced form.
  • Statistics: lmer from lmerTest in R.
    • WER was predicted from MFA Status, AgeGroup and Gender.
    • Target Word and Speaker Id were random effects, with random slopes.
    • Models were fitted separately for overall, CCR-only and ING-only data, each with and without the LM.
    • Code and data are released on OSF.

Interesting ideas

  • I1. Instead of hand-coding dialect features, the authors gave the aligner both canonical and reduced pronunciations and let it choose, which yields token-level variant labels at scale.
  • I2. Errors are attributed to a named cause only when an explicit, checkable test passes, here "the output is a one-edit neighbor of the realized form".
  • I3. Neighbors are computed from what was actually pronounced, not from the dictionary form, which better models what the recognizer had to work with.
  • I4. Running the same system with and without its context component isolates how much context rescues acoustically ambiguous items.
  • I5. Mixed-effects models with random effects for item and speaker stop frequent words or prolific speakers from driving the conclusion.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): the en lexicon could list non-standard surface variants (e.g. "goin'", "talkin'") under one lemma with a variant feature. The tagger could then record which variant occurred instead of treating it as OOV.
  • I2 (learn): parser and tagger errors in evaluation could be binned by similar mechanical tests, such as near-homograph confusion or attachment to an adjacent head. This would make error reports explainable rather than just aggregate scores.
  • I3 (tokens): normalization of misspelled or dialectal tokens could generate candidate lemmas by edit distance from the observed surface string, ranked by context.
  • I4 (learn): ablating the frame or world-knowledge layer on the same inputs would show where context actually repairs tree or tag errors, as a built-in control.
  • I5 (learn): per-item and per-source grouping in evaluation would keep a few very frequent verbs or a single treebank from masking real differences between parsers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksSpeech & spoken language · 480 works

Cite

Cite as Hamid Mojarad and Kevin Tang. 2025. Automatic Speech Recognition of African American English: Lexical and Contextual Effects. arXiv:2506.06888 · Proc. Interspeech 2025 (2025) 3883-3887.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.21437/interspeech.2025-1511 · arXiv 2506.06888

@misc{mojarad2025automatic,
  title = {{Automatic Speech Recognition of African American English: Lexical and Contextual Effects}},
  author = {Hamid Mojarad and Kevin Tang},
  year = {2025},
  doi = {10.21437/interspeech.2025-1511},
  eprint = {2506.06888},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.21437/interspeech.2025-1511}
}

References

31 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Koenecke, A. Nam, E. Lake, J. Nudell, M. Quartey, Z. Mengesha, C. Toups, J. R. Rickford, D. Jurafsky, and S. Goel, “Racial disparities in automated speech recognition,” Proceedings of the National Academy of Sciences, vol. 117, no. 14, pp. 7684–7689, 2020. ↗
  2. J. L. Martin and K. Tang, “Understanding racial disparities in automatic speech recognition: The case of habitual “be”,” in Interspeech 2020, 2020, pp. 626–630. find
  3. A. B. Wassink, C. Gansen, and I. Bartholomew, “Uneven success: automatic speech recognition and ethnicity-related dialects,” Speech Communication, vol. 140, pp. 50–70, 2022. ↗
  4. E. R. Thomas and G. Bailey, “Segmental phonology of African American English,” in The Oxford Handbook of African American Language. Oxford University Press, 07 2015. find
  5. R. Gregová, “A comparative analysis of consonant clusters in English and in Slovak,” Bulletin of the Transilvania University of Brasov. Series IV: Philology and Cultural Studies, pp. 79–84, 2010. find
  6. T. Kendall, C. Vaughn, C. Farrington, K. Gunter, J. McLean, C. Tacata, and S. Arnson, “Considering performance in the automated and manual coding of sociolinguistic variables: Lessons from variable (ING),” Frontiers in Artificial Intelligence, vol. 4, 2021. ↗
  7. P. A. Luce and D. B. Pisoni, “Recognizing spoken words: The neighborhood activation model,” Ear and Hearing, vol. 19, no. 1, pp. 1–36, 1998. ↗
  8. P. Jyothi and K. Livescu, “Revisiting word neighborhoods for speech recognition,” in Proceedings of the 2014 Joint Meeting of SIGMORPHON and SIGFSM, Ö. Çetinoğlu, J. Heinz, A. Maletti, and J. Riggle, Eds. Baltimore, Maryland: Association for Computational Linguistics, Jun. 2014, pp. 1–9. find
  9. S. Goldwater, D. Jurafsky, and C. D. Manning, “Which words are hard to recognize? prosodic, lexical, and disfluency factors that increase speech recognition error rates,” Speech Communication, vol. 52, no. 3, pp. 181–200, 2010. ↗
  10. V. Marian, J. Bartolotti, S. Chabal, and A. Shook, “Clearpond: Cross-linguistic easy-access resource for phonological and orthographic neighborhood densities,” PLOS ONE, vol. 7, no. 8, pp. 1–11, 08 2012. ↗
  11. K. Huang, A. Zhang, Z. Yang, P. Guo, B. Mu, T. Xu, and L. Xie, “Contextualized end-to-end speech recognition with contextual phrase prediction network,” in Interspeech 2023, 2023, pp. 4933– 4937. find
  12. N. Manh Tien Anh and T. Ho Sy, “Improving speech recognition with prompt-based contextualized ASR and LLM-based re-predictor,” in Interspeech 2024, 2024, pp. 737–741. find
  13. J. Fox and N. Delworth, “Improving contextual recognition of rare words with an alternate spelling prediction model,” in Interspeech 2022, 2022, pp. 3914–3918. find
  14. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the middle: How language models use long contexts,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 157–173, 2024. ↗
  15. R. G. Podlubny, T. M. Nearey, G. Kondrak, and B. V. Tucker, “Assessing the importance of several acoustic properties to the perception of spontaneous speech,” The Journal of the Acoustical Society of America, vol. 143, no. 4, pp. 2255–2268, 04 2018. 5 https://credit.niso.org/ ↗
  16. Y. Tang and A. K. H. Tung, “Contextualized speech recognition: Rethinking second-pass rescoring with generative large language models,” in Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, K. Larson, Ed. International Joint Conferences on Artificial Intelligence Organization, 8 2024, pp. 6478–6485. find
  17. T. Kendall and C. Farrington, “The Corpus of Regional African American Language,” 2023, publisher: The Online Resources for African American Language Project. [Online]. Available: https://oraal.uoregon.edu/coraal ↗
  18. M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, “Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi,” in Proc. Interspeech 2017, 2017, pp. 498–502. find
  19. A. Baevski, H. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: a framework for self-supervised learning of speech representations,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, ser. NIPS ’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
  20. K. Heafield, “KenLM: Faster and smaller language model queries,” in Proceedings of the Sixth Workshop on Statistical Machine Translation, C. Callison-Burch, P. Koehn, C. Monz, and O. F. Zaidan, Eds. Edinburgh, Scotland: Association for Computational Linguistics, Jul. 2011, pp. 187–197.
  21. P. A. Luce, “Neighbourhoods of words in the mental lexicon,” Indiana University, Bloomington, IN, Tech. Rep. Technical Report No. 6, 1986. find
  22. V. I. Levenshtein, “Binary codes capable of correcting deletions, insertions, and reversals,” in Soviet Physics Doklady, vol. 10, no. 8, 1966, pp. 707–710. find
  23. R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2023. [Online]. Available: https://www.R-project.org/ ↗
  24. A. Johnson, C. Chance, K. Stiemke, H. Veeramani, N. B. Shankar, and A. Alwan, “An analysis of large language models for African American English speaking children’s oral language assessment,” Journal of Black Excellence in Engineering, Science, & Technology, vol. 1, dec 4 2023. find
  25. K. Chang, Y.-H. Chou, J. Shi, H.-M. Chen, N. Holliday, O. Scharenborg, and D. R. Mortensen, “Self-supervised speech representations still struggle with African American Vernacular English,” in Interspeech 2024, 2024, pp. 4643–4647.
  26. P. Gurunath Shivakumar and P. Georgiou, “Transfer learning from adult to children for speech recognition: Evaluation, analysis and recommendations,” Computer Speech & Language, vol. 63, p. 101077, 2020. ↗
  27. S. Kwon, S.-J. Kim, and J. Y. Choeh, “Preprocessing for elderly speech recognition of smart devices,” Computer Speech & Language, vol. 36, pp. 110–121, 2016. ↗
  28. R. Tatman and C. Kasten, “Effects of talker dialect, gender and race on accuracy of Bing Speech and YouTube automatic captions,” in Interspeech 2017, 2017, pp. 934–938. find
  29. C. Harris, C. Mgbahurike, N. Kumar, and D. Yang, “Modeling gender and dialect bias in automatic speech recognition,” in Findings of the Association for Computational Linguistics: EMNLP 2024, Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, pp. 15 166–15 184.
  30. A. Graves and N. Jaitly, “Towards end-to-end speech recognition with recurrent neural networks,” in Proceedings of the 31st International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, E. P. Xing and T. Jebara, Eds., vol. 32, no. 2. Bejing, China: PMLR, 22–24 Jun 2014, pp. 1764– 1772. find
  31. R. Porwal, A. Rozet, J. Gowda, P. Houck, K. Tang, and S. Moeller, “Analysis of LLM as a grammatical feature tagger for African American English,” in Findings of the Association for Computational Linguistics: NAACL 2025, L. Chiruzzo, A. Ritter, and L. Wang, Eds. Albuquerque, New Mexico: Association for Computational Linguistics, Apr. 2025, pp. 7744–7756.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mojarad-2025-automatic-speech-recognition-african-american/