Elamur Research⌕
☰
home / papers / 2026 / Phonologically-aware Automatic Speech Recognition Evaluation of Low-Resource Languages: The Case of Basque Dialects

Phonologically-aware Automatic Speech Recognition Evaluation of Low-Resource Languages: The Case of Basque Dialects

Paper · 2026 · Christoforos Souganidis, Asier Herranz, Ibon Saratxaga et al.

Summary

Speech recognizers for Basque are mostly built and judged on standard language, so it is unclear how well they cope with the strong regional variation of this low-resource isolate. The authors provide what they describe as the first dialect-sensitive ASR evaluation for Basque on spontaneous speech. They use a proprietary 140-hour collection of EITB television programmes. Each speaker carries a dialect label, and every utterance has two transcriptions: a standardized one and a pseudo-phonetic one.

They decoded the data with NVIDIA NeMo's publicly released stt_eu_conformer_transducer_large_v2 model and compared word error rates across dialects with non-parametric statistics. Standard Basque was recognized significantly better than every dialect, and Gipuzkoan proved the hardest variety.

To explain these gaps, the authors examined single-word substitution errors against the pseudo-phonetic transcripts. This analysis produced a catalogue of 52 recurring phonological phenomena. Utterances containing more of these phenomena tended to have higher error rates, though the link was weak.

The work argues that evaluations which ignore dialect hide real performance gaps. It also supports building ASR for diverse low-resource languages with linguistic knowledge in mind.

Key points

  • Corpus composition: the EITB corpus covers Standard Basque (924 speakers, ~59 h), Gipuzkoan (547 speakers, ~33 h), Biscayan (335 speakers, ~19 h), Upper Navarrese (~2 h), Lapurdian (~1 h) and Lower Navarrese (a single speaker, 8 utterances).
  • Model training data: the evaluated Conformer-Transducer had been trained on 772 h of Basque. This included 95.75 h of spontaneous radio speech and some dialectal material, for example 19 h of Biscayan and 11.7 h of Gipuzkoan.
  • Statistical result: a Kruskal-Wallis test showed a strong dialect effect on WER (H(5) = 6630.784, p < .001). Dunn's post-hoc tests with Holm correction confirmed that the standard variety outperformed all dialects.
  • Lower Navarrese caveat: this dialect had a numerically higher WER, but too little data for the difference to be statistically reliable.
  • Phonological phenomena: the 52 recurrent phenomena fall into deletions, epentheses and substitutions. Examples include intervocalic and coda deletion, consonant epenthesis, vowel changes, palatalization and voicing assimilation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of the Basque CountryLeioa · Spain57 works · 16,520 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 3Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 worksSpeech & spoken language · 480 worksLow-resource languages · 267 works

Cite

Cite as Christoforos Souganidis, Asier Herranz, Ibon Saratxaga, Eva Navas and Inma Hernaez. 2026. Phonologically-aware Automatic Speech Recognition Evaluation of Low-Resource Languages: The Case of Basque Dialects. Proceedings of the First Workshop on Dialects in NLP –- A Resource Perspective.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/262fznwr54us

@inproceedings{souganidis2026phonologically,
  title = {{Phonologically-aware Automatic Speech Recognition Evaluation of Low-Resource Languages: The Case of Basque Dialects}},
  author = {Christoforos Souganidis, Asier Herranz, Ibon Saratxaga, Eva Navas and Inma Hernaez},
  booktitle = {Proceedings of the First Workshop on Dialects in NLP –- A Resource Perspective},
  year = {2026},
  doi = {10.63317/262fznwr54us},
  url = {https://doi.org/10.63317/262fznwr54us}
}

References

35 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Md Mahfuz Ibn Alam, Sina Ahmadi, and Antonios Anastasopoulos. 2024. CODET: A Benchmark for Contrastive Dialectal Evaluation of Machine Translation. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1790–1859, St. Julian’s, Malta. Association for Computational Linguistics… find
  2. Varun deCastro Arrazola, Edoardo Cavirani, Kathrin Linke, and Francesc Torres-Tamarit. 2015. A typological study of vowel interactions in Basque. Isogloss, 2015(1/2):147–177. find
  3. Gotzon Aurrekoetxea, Ane Arandia, Malalen Camino, and Xantiana Etchebest. 2017. Dialectología perceptual del vasco: aplicación del Little Arrow Method. In Florentino Paredes García, Ana M. Cestero Mancera, and Isabel Molina Martos, editors, Investigaciones actuales en Lingüística. Vol. V: Sobre vari… find
  4. Ainara Estarrona, Izaskun Etxeberria, Ricardo Etxepare, Manuel Padilla-Moyano, and Ander Soraluze. 2020. Dealing with dialectal variation in the construction of the Basque historical corpus. In Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects, pages 79–89, Barcelo… find
  5. Gotzon Aurrekoetxea, Iñaki Gaminde, José Luis Or-maetxea, and Xarles Videgain. 2019. Euskalkien Sailkapen Berria [New Classification of Basque Dialects]. UPV/EHU, Bilbao, Spain. find
  6. Fahim Faisal, Orevaoghene Ahia, Aarohi Srivastava, Kabir Ahuja, David Chiang, Yulia Tsvetkov, and Antonios Anastasopoulos. 2024. DIALECT-BENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages. arXiv preprint. find
  7. Ekhi Azurmendi, Joseba Fernandez de Landa, Jaione Bengoetxea, Maite Heredia, Julen Etxaniz, Mikel Zubillaga, and Ander Soraluze. 2025. BERnaT: Basque Encoders for Representing Natural Textual Diversity. arXiv preprint. find
  8. Joseba Fernandez de Landa. 2019. Large Scale Linguistic Processing of Tweets to Understand Social Interactions among Speakers of Less Resourced Languages: The Basque Case. Information, 10(6). find
  9. Basque Government. 2016. VI. Inkesta Soziolinguistikoa Euskal Autonomia Erkidegoa [6. find
  10. Inma Hernaez, Iker Luengo, Eva Navas, Maren Zubizarreta, Iñaki Gaminde, and Jon Sanchez. 2003. The basque speech_dat (II) database: a description and first test recognition results. In 8th European Conference on Speech Communication and Technology (Eurospeech 2003), pages 1549–1552. find
  11. Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences of the United States of America, 117(14):7684–7689. find
  12. Saioa Larraza, Arthur G. Samuel, and Miren Lourdes Oñederra. 2017. Where do dialectal effects on speech processing come from? Evidence from a cross-dialect investigation. Quarterly Journal of Experimental Psychology, 70(1):92–108. find
  13. Arthur Hinsvark, Natalie Delworth, Miguel Del Rio, Quinten McNamara, Joshua Dong, Ryan Westerman, Ilya Pirkin, and Nishchal Bhandari. 2021. Accented Speech Recognition: A Survey. arXiv preprint. find
  14. Joshua L. Martin and Kevin Tang. 2020. Understanding Racial Disparities in Automatic Speech Recognition: The Case of Habitual “be”. In Interspeech 2020, pages 626–630. find
  15. Dirk Hovy and Diyi Yang. 2021. The Importance of Modeling Social Factors of Language: Theory and Practice. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 588–602, Online. Association for Computa…
  16. Igor Odriozola, Inma Hernaez, María Inés Torres, Luis Javier Rodriguez-Fuentes, Mikel Peñagarikano, and Eva Navas. 2014. Basque speecon-like and Basque SpeechDat MDB-600: speech databases for the development of ASR technology for Basque. In Proceedings of the Ninth International Conference on Langua… find
  17. José Ignacio Hualde. 1991. Basque Phonology. Routledge. find
  18. José Ignacio Hualde and Iñaki Gaminde. 1998. On the phonology and morphology of Basque dialects. Anuario del Seminario de Filología Vasca "Julio de Urquijo", 32:71–108. find
  19. Mikel Peñagarikano, Amparo Varona, Germán Bor-dél, and Luis Javier Rodriguez-Fuentes. 2023. Semisupervised Speech Data Extraction from Basque Parliament Sessions and Validation on Fully Bilingual Basque–Spanish ASR. Applied Sciences, 13(14):848492. find
  20. José Ignacio Hualde, Oihana Lujanbio, and Juan Joxe Zubiri. 2010. Goizueta Basque. Journal of the International Phonetic Association, 40(1):113–127. find
  21. Alec Radford, Jong Wook Wu, Dario Amodei, et al. 2023. Whisper: Robust Speech Recognition via Large-Scale Weak Supervision. arXiv preprint. ArXiv:2212.04356 [cs.CL].
  22. José Ignacio Hualde and Jon Ortiz de Urbina. 2003. A Grammar of Basque. Mouton de Gruyter. find
  23. José Ignacio Hualde and Koldo Zuazo. 2007. The standardization of the Basque language. Language Problems and Language Planning, 31(2):143–168. find
  24. Jiatong Shi, Shih-Heng Wang, William Chen, Martijn Bartelds, Vanya Bannihatti Kumar, Jinchuan Tian, Xuankai Chang, Dan Jurafsky, Karen Livescu, Hung yi Lee, and Shinji Watanabe. 2024. ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets. In Inte… find
  25. Xuedong Huang, Federico Alleva, Hsiao-Wuen Hon, Wei Hwang, and Edward Reichenbach. 2004. Spoken language processing: A guide to theory, algorithm, and system development. International Journal of Speech Technology, 7(2):111–126. find
  26. Rachael Tatman. 2017. “oh, I’ve Heard That Before”: Modelling Own-Dialect Bias After Perceptual Learning by Weighting Training Data. In Proceedings of the 7th Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2017), pages 29–34, Valencia, Spain. Association for Computational Linguis… find
  27. JASP Team. 2026. JASP (Version 0.19.1.0) [Computer software]. Accessed: 2026-02-24. find
  28. Aditya Joshi, Raj Dabre, Diptesh Kanojia, Zhuang Li, Haolan Zhan, Gholamreza Haffari, and Doris Dippold. 2025. Natural Language Processing for Dialects of a Language: A survey. ACM Computing Surveys, 57(6):149:1–149:37. find
  29. Larraitz Uria and Ricardo Etxepare. 2012. Hizk-eren arteko aldakortasun sintaktikoa aztertzeko metodologiaren nondik norakoak: BASYQUE ap-likazioa. Lapurdum. Euskal ikerketen aldizkaria \| Revue d’études basques \| Revista de estudios vascos \| Basque studies review, (16):117–135. find
  30. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R. Rickford, Dan Jurafsky, and Socrates Vakirtzian, Vivian Stamou, Yannis Kazos, and Stella Markantonatou. 2025. Dialectal treebanks and their relation with the standard variety: The case of East… find
  31. Alicia Beckford Wassink, Cady Gansen, and Isabel Bartholomew. 2022. Uneven success: automatic speech recognition and ethnicity-related dialects. Speech Communication, 140:50–70. find
  32. Xuesong Yang, Kartik Audhkhasi, Andrew Rosenberg, Samuel Thomas, Bhuvana Ramabhadran, and Mark Hasegawa-Johnson. 2018. Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), page 1–5. IEE…
  33. Marcos Zampieri, Preslav Nakov, and Yves Scherrer. 2020. Natural language processing for similar languages, varieties, and dialects: A survey. Natural Language Engineering, 26(6):595–612. find
  34. Marcos Zampieri, Preslav Nakov, and Yves Scherrer. 2024. Natural language processing for similar languages, varieties, and dialects: A survey. Natural Language Engineering, 26(6):595–614. find
  35. Koldo Zuazo. 2008. Euskalkiak. Euskararen dialektoak. Elkar. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/souganidis-2026-phonologically-aware-automatic-speech-recognition/