Elamur Research⌕
☰
home / papers / 2026 / A Fine-tuned ASR Model for Historical American Dialect Recordings

A Fine-tuned ASR Model for Historical American Dialect Recordings

Paper · 2026 · Steven Coats

Summary

Off-the-shelf ASR systems such as Whisper perform poorly on conversational, dialectal and historical speech, so thousands of hours of carefully sampled American dialect fieldwork remain untranscribed. Coats addresses this by turning DASS2019, the manually transcribed sample of the Linguistic Atlas of the Gulf States interviews, into a training-ready resource called DASS2019_NLP. It contains 344 hours of Southern conversational speech recorded 1969–1983. The author cleaned the transcription markup, cut the recordings into segments of up to 30 seconds, and fine-tuned six Whisper models from tiny to large-v3. Evaluation used three test sets: held-out DASS data, a CORAAL subset, and a Common Voice subset. In-domain WER fell by about 37% on average. The fine-tuned models also transcribed African American English in CORAAL more accurately. On read Common Voice speech the original OpenAI models generally stayed ahead. The dataset and the best model are public, which opens large-scale quantitative work on historical Southern and African American English.

Key points

  • DASS2019_NLP is built from 408 recordings of 64 main informants, plus auxiliary speakers such as spouses and interviewers. It yields 48,214 segments with a mean length of 25.69 s, resampled to 16 kHz.
  • Six models were fine-tuned: Whisper tiny, base, small, medium, large-v2 and large-v3. How much fine-tuning helped did not depend much on model size.
  • The fine-tuned models reduced in-domain WER by about 37% on average and also gained on CORAAL. On Common Voice the baselines were usually better, which the author attributes to the mismatch between read and conversational speech.
  • Two resources were released on Hugging Face: the dataset (stcoats/DASS2019_NLP) and the model whisper-large-v3-DASS2019-ct2.
  • About 5,000 hours of LAGS audio alone remain untranscribed, and other Linguistic Atlas Project collections hold many thousands more hours.

Interesting ideas

  • I1. The markup was cleaned by category: unintelligible, non-speech, non-word and comment codes were deleted, doubtful passages kept their text without the tag, and the privacy-beep code was preserved as a token. Manual spot checks also caught "?" being used informally for unintelligible speech.
  • I2. Training units were built by greedily merging consecutive turns from the same speaker up to a length cap, so no segment mixes speakers.
  • I3. Each model was tested on three sets: in-domain data, a related but different variety, and a style-mismatched set where gains were not expected. The last set reveals whether improvement is real adaptation or just overfitting to the domain.
  • I4. Every segment points back to a recording and an informant, so demographic metadata can be joined as a lookup table for analysis by region, age or ethnicity.
  • I5. Transcriber-flagged doubtful passages were kept as labels rather than discarded, which trades some label certainty for more data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Converting dialect or noisy treebanks to UD could use a similar per-annotation-type policy, deciding explicitly what to drop, unwrap or keep, instead of stripping all markup blindly.
  • I2 (frames): Coreference and speech-act training data from dialogues could be chunked the same way, keeping each unit single-speaker so who said what stays unambiguous.
  • I3 (learn): Each retraining of the tagger or parser could be scored on in-domain, near-domain and deliberately mismatched genres, with the mismatched set serving as a negative control.
  • I4 (learn): Silver-data sentences could carry source IDs linked to genre or dialect metadata, so parser errors can be broken down by variety rather than averaged away.
  • I5 (learn): Silver parses where witness parsers partly disagree could be kept with a doubt flag, letting training weight them lower instead of throwing them out.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 480 works

Cite

Cite as Steven Coats. 2026. A Fine-tuned ASR Model for Historical American Dialect Recordings. Proceedings of the Fifteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5bjeqct6ozd3

@inproceedings{coats2026fine,
  title = {{A Fine-tuned ASR Model for Historical American Dialect Recordings}},
  author = {Steven Coats},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5bjeqct6ozd3},
  url = {https://doi.org/10.63317/5bjeqct6ozd3}
}

References

39 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Harold B. Allen, editor. 1973–6. The Linguistic Atlas of the Upper Midwest. 3 vols. University of Minnesota Press, Minneapolis. find
  2. Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber. 2020. Common Voice: A massively-multilingual speech corpus. ↗
  3. Doug Biber. 1993. Representativeness in Corpus Design. Literary and Linguistic Computing, 8(4):243–257. ↗
  4. Hervé Bredin. 2023. Pyannote.audio 2.1 speaker diarization pipeline: Principle, benchmark, and recipe. In INTERSPEECH 2023, pages 1983– 1987. find
  5. Keiko Bridwell and Margaret E. L. Renwick. 2024. Race, place, and education: Charting the wine-whine merger in the U.S. South. American Speech, 99(4):441–467. find
  6. Kalvin Chang, Yi-Hui Chou, Jiatong Shi, Hsuan-Ming Chen, Nicole Holliday, Odette Scharenborg, and David R. Mortensen. 2024. Self-supervised speech representations still struggle with African American Vernacular English. In Interspeech 2024, pages 4643–4647.
  7. Luyao Cheng, Hui Wang, Chong Deng, Siqi Zheng, Yafeng Chen, Rongjie Huang, Qinglin Zhang, Qian Chen, Xihao Li, and Wen Wang. 2025. Integrating audio, visual, and semantic information for enhanced multimodal speaker diarization on multi-party conversation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 19914–19928, Vienna, Austria. Association for Computational Linguistics. ↗
  8. Lisa J. Green. 2002. African American English: A linguistic introduction. Cambridge University Press, Cambridge. find
  9. Jack Grieve, Sara Bartl, Matteo Fuoli, Jason Grafmiller, Weihang Huang, Alejandro Jawerbaum, Akira Murakami, Marcus Perlman, Dana Roemling, and Bodo Winter. 2025. The sociolinguistic foundations of language modeling. Frontiers in Artificial Intelligence, 7:1472411. Publisher: Frontiers. ↗
  10. Tyler Kendall. 2007. Enhancing sociolinguistic data collections: The North Carolina Sociolinguistic Archive and Analysis Project. Penn Working Papers in Linguistics, 13(2):15–26. find
  11. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R. Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684– 7689. ↗
  12. William A. Kretzschmar, Margaret E. L. Renwick, Lisa M. Lipani, Michael L. Olsen, Rachel M. Olsen, Yuanming Shi, and Joseph A. Stanley. 2019. Transcriptions of the Digital Archive of Southern Speech. find
  13. Per E. Kummervold, Javier de la Rosa, Freddy Wetjen, Rolv-Arild Braaten, and Per Erik Solberg. 2024. Whispering in Norwegian: Navigating orthographic and dialectic challenges. In Interspeech 2024, pages 3984–3988. find
  14. Hans Kurath, Marcus L. Hansen, Bernard Bloch, and Julia Bloch, editors. 1939–43, reprinted 1972. Linguistic Atlas of New England. 3 vols. Brown University Press, Providence, RI. find
  15. William Labov, Sharon Ash, and Charles Boberg. 2008. The Atlas of North American English: Phonetics, Phonology and Sound Change. De Gruyter Mouton. find
  16. Colin Lea, Zifang Huang, Jaya Narain, Lauren Tooley, Dianna Yee, Dung Tien Tran, Panayiotis Georgiou, Jeffrey P Bigham, and Leah Findlater. 2023. From user perceptions to technical improvement: Enabling people who stutter to better use speech recognition. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, New York, NY, USA. Association for Computing Machinery. find
  17. Lucas Maison and Yannick Esteve. 2023. Improving accented speech recognition with multi-domain training. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  18. Raven I. McDavid and Raymond K. O’Cain, editors. 1980. Linguistic Atlas of the Middle and South Atlantic States, Fascicles 1–2. University of Chicago Press, Chicago. find
  19. Hamid Mojarad and Kevin Tang. 2025. Automatic Speech Recognition of African American English: Lexical and Contextual Effects. In Interspeech 2025, pages 3883–3887. ↗
  20. Dong Nguyen, A. Seza Doğruöz, Carolyn P. Rosé, and Franciska De Jong. 2016. Computational Sociolinguistics: A Survey. Computational Linguistics, 42(3):537–593. ↗
  21. Rachel M. Olsen, Michael L. Olsen, and Margaret E. L. Renwick. 2018. The impact of sub-region on /ai/ weakening in the U.S. South. Proceedings of Meetings on Acoustics, 31(1):060005. find
  22. Rachel M. Olsen, Michael L. Olsen, Joseph A. Stanley, Margaret E. L. Renwick, and William Kretzschmar. 2017. Methods for transcription and forced alignment of a legacy speech corpus. Proceedings of Meetings on Acoustics, 30(1):060001. find
  23. Lee Pederson. 1985. A matrix for word geography (LAGS working papers, 3rd series, introduction). find
  24. Lee Pederson, Susan L. McDaniel, and Carol M. Adams, editors. 1986–92. Linguistic Atlas of the Gulf States. 7 vols. University of Georgia Press, Athens, Georgia. find
  25. Yifan Peng, Muhammad Shakeel, Yui Sudo, William Chen, Jinchuan Tian, Chyi-Jiunn Lin, and Shinji Watanabe. 2025. OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning. In Interspeech 2025, pages 2225–2229. find
  26. Krishna C. Puvvada, Piotr Żelasko, He Huang, Oleksii Hrinchuk, Nithin Rao Koluguri, Kunal Dhawan, Somshubra Majumdar, Elena Rastorgueva, Zhehuai Chen, Vitaly Lavrukhin, Jagadeesh Balam, and Boris Ginsburg. 2024. Less is More: Accurate Speech Recognition Translation without Web-Scale Data. In Interspeech 2024, pages 3964–3968. find
  27. Yanmin Qian, Xun Gong, and Houjun Huang. 2022. Layer-wise fast adaptation for end-to-end multi-accent speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:2842–2853. ↗
  28. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. ↗
  29. Margaret E. L. Renwick and Joseph A. Stanley. 2017. Static and dynamic approaches to vowel shifting in the Digital Archive of Southern Speech. Proceedings of Meetings on Acoustics, 30(1):060003. find
  30. John Rickford. 1999. African American Vernacular English: Features, evolution, educational implications. Wiley. find
  31. George Saon, Avihu Dekel, Alexander Brooks, Tohru Nagano, Abraham Daniels, Aharon Satt, Ashish Mittal, Brian Kingsbury, David Haws, Edmilson Morais, Gakuto Kurata, Hagai Aronowitz, Ibrahim Ibrahim, Jeff Kuo, Kate Soule, Luis Lastras, Masayuki Suzuki, Ron Hoory, Samuel Thomas, Sashi Novitasari, Takashi Fukuda, Vishal Sunder, Xiaodong Cui, and Zvi Kons. 2025. Granite-speech: Open-source speech-aware LLMs with strong English ASR capabilities. find
  32. Clement Sicard, Kajetan Pyszkowski, and Victor Gillioz. 2023. Spaiche: Extending state-of-the-art ASR models to Swiss German dialects. ↗
  33. Erik R. Thomas. 2007. Phonological and phonetic characteristics of African American Vernacular English. Language and Linguistics Compass, 1(5):450–475. ↗
  34. Melissa Torgbi, Andrew Clayman, Jordan J. Speight, and Harish Tayyar Madabushi. 2025. Adapting Whisper for regional dialects: Enhancing public services for vulnerable populations in the United Kingdom. ↗
  35. Leonora Vesterbacka, Faton Rekathati, Robin Kurtz, Justyna Sikora, and Agnes Toftgård. 2025. Swedish Whispers; Leveraging a Massive Speech Corpus for Swedish Speech Recognition. In Interspeech 2025, pages 758–762. ↗
  36. Quan Wang, Yiling Huang, Guanlong Zhao, Evan Clark, Wei Xia, and Hank Liao. 2024. DiarizationLM: Speaker Diarization Post-Processing with Large Language Models. In Interspeech 2024, pages 3754–3758. find
  37. Walt Wolfram and Natalie Schilling. 2015. American English: Dialects and variation, 3rd Edition. John Wiley & Sons. 9. find
  38. Language Resource References Tyler Kendall and Charlie Farrington. 2023. The Corpus of Regional African American Language. PID https://doi.org/10.7264/1ad5-6t35. ↗
  39. William A. Kretzschmar and Paulina Bounds and Jacqueline Hettel and Steven Coats and Lee Pederson and Lisa Lena Opas-Hänninen and Ilkka Juuso and Tapio Seppänen. 2012. Digital Archive of Southern Speech. Linguistic Data Consortium. PID https://doi.org/10.35111/5bnt-r659. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/coats-2026-fine-tuned-asr-model-historical/