Elamur Research⌕
☰
home / papers / 2026 / What Do Neural Speech Models Know About Phonology? Evidence from Structured Phoneme Confusions

What Do Neural Speech Models Know About Phonology? Evidence from Structured Phoneme Confusions

Paper · 2026 · Eli Stafford, Aimée Lahaussois and Guillaume Wisniewski

Summary

Phoneme-level ASR evaluation usually treats each phoneme as an unrelated symbol, so it cannot show which aspects of a sound a model fails on. The authors instead describe phonemes as bundles of distinctive phonological features (voicing, place, manner, vowel quality and so on). They examine only the outputs of transformer phoneme recognisers, without looking inside the models. The main system is the multilingual Wav2Vec2Phoneme model, evaluated on 12 languages with eSpeak-generated references. A second, language-specific recogniser for Thulung, trained on expert phonemic annotations, checks that the results are not side effects of automatic phonemisation.

The framework has three parts: feature-based distances to test whether errors stay close to their targets, analysis of which features are gained or lost in errors, and effect sizes assessed using bootstrapped confidence intervals together with ranges within which differences are treated as practically negligible. Substitution errors turn out to be phonologically local. They also show consistent directional asymmetries: broad class and sonority features tend to be added, while detailed place, manner and secondary articulation features tend to be dropped. This links model diagnosis to phonological theory and to similar feature asymmetries found in human speech perception.

Key points

  • Models: The main model is Wav2Vec2Phoneme (Xu et al., 2022). It is an XLSR-53 encoder fine-tuned with CTC on Common Voice and Babel, using eSpeak phonemisation, and accessed through Hugging Face.
  • Control setting: A separate Thulung recogniser was trained on expert transcriptions from the Pangloss Collection. Thulung is an endangered Sino-Tibetan (Tibeto-Burman) language of Nepal.
  • Scope: The 12 evaluated languages have phoneme inventories of 29 to 61 phonemes. PER is reported as a sanity check, and only phoneme-to-phoneme substitutions are analysed; insertions, deletions and control symbols are excluded.
  • Findings: Confused phonemes are much closer in feature space than chance would predict. Broad class and sonority features shift toward positive values in errors, while fine-grained place, manner and secondary features shift toward negative values.
  • Limitations:
    • The analysis describes errors only after they occur, not the likelihood of an error.
    • It relies on a single model family.
    • The results depend on the soundvectors feature inventory.
    • Rare features, such as temporal or diphthongal properties, are analysed separately.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🎓Université Grenoble AlpesSaint-Martin-d'Hères · France20 works · 8,059 citations🔬Laboratoire d'Informatique de GrenobleGrenoble · France21 works · 688 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

Cite

Cite as Eli Stafford, Aimée Lahaussois and Guillaume Wisniewski. 2026. What Do Neural Speech Models Know About Phonology? Evidence from Structured Phoneme Confusions. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1288

@inproceedings{stafford2026what,
  title = {{What Do Neural Speech Models Know About Phonology? Evidence from Structured Phoneme Confusions}},
  author = {Eli Stafford, Aimée Lahaussois and Guillaume Wisniewski},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1288},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1288}
}

References

26 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Evangelia Adamou, Séverine Guillaume, and Alexis Michaud. 2025. The Pangloss Collection: Opening up research data on endangered and under-documented languages. Language, 101(1):e38–e59. find
  2. Afra Alishahi, Marie Barking, and Grzegorz Chrupała. 2017. Encoding of phonology in a recurrent neural model of grounded speech. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 368–378, Vancouver, Canada. Association for Computational Linguistics. find
  3. Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: a framework for self-supervised learning of speech representations. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. Curran Associates Inc.
  4. Yonatan Belinkov. 2022. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207–219.
  5. Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72.
  6. Noam Chomsky and Morris Halle. 1968. The Sound Pattern of English. Harper & Row. find
  7. G. N. Clements and Elizabeth Hume. 1995. The internal organization of speech sounds. In John Goldsmith, editor, The Handbook of Phonological Theory. Blackwell. find
  8. William G. Cochran. 1954. The combination of estimates from different experiments. Biometrics, 10(1):101–129. find
  9. Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2021. Unsupervised cross-lingual representation learning for speech recognition. In Interspeech 2021, pages 2426–2430. find
  10. Anne Cutler, Andrea Weber, Roel Smits, and Nicole Cooper. 2004. Patterns of english phoneme confusions by native and non-native listeners. Journal of the Acoustical Society of America, 116(6):3668–3678. find
  11. Marianne de Heer Kloots, Hosein Mohebbi, Charlotte Pouw, Gaofei Shen, Willem Zuidema, and Martijn Bentum. 2025. What do self-supervised speech models know about Dutch? Analyzing advantages of language-specific pre-training. In Interspeech 2025, pages 256–260. find
  12. Maureen de Seyssel, Marvin Lavechin, Yossi Adi, Emmanuel Dupoux, and Guillaume Wisniewski. 2022. Probing phoneme, language and speaker information in unsupervised speech representations. In Interspeech 2022, pages 1402–1406. find
  13. Bradley Efron and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. Chapman and Hall/CRC, New York. find
  14. Steven Greenberg and Shuangyu Chang. 2000. Linguistic dissection of switchboard-corpus automatic speech recognition systems. In Proc. ASR2000 - Automatic Speech Recognition: Challenges for the New Millenium, pages 195–202. find
  15. Séverine Guillaume, Guillaume Wisniewski, Cécile Macaire, Guillaume Jacques, Alexis Michaud, Benjamin Galliot, Maximin Coavoux, Solange Rossato, Minh-Châu Nguyên, and Maxime Fily. 2022. Fine-tuning pre-trained models for automatic speech recognition, experiments on a fieldwork corpus of japhug (tran… find
  16. Julian P. T. Higgins and Simon G. Thompson. 2002. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11):1539–1558. find
  17. Roman Jakobson, Gunnar Fant, and Morris Halle. 1963. Preliminaries to Speech Analysis: The Distinctive Features and Their Correlates. MIT Press. find
  18. Daniël Lakens. 2017. Equivalence tests: A practical primer for t tests, correlations, and meta-analyses. Social Psychological and Personality Science, 8(4):355–362. find
  19. Kenneth O. McGraw and S. P. Wong. 1992. A common language effect size statistic. Psychological Bulletin, 111(2):361–365. find
  20. Bernd T. Meyer, Matthias Wächter, Thomas Brand, and Birger Kollmeier. 2007. Phoneme confusions in human and automatic speech recognition. In Interspeech 2007, pages 1485–1488. find
  21. Jeff Mielke. 2008. The Emergence of Distinctive Features. Oxford University Press. find
  22. George A. Miller and Patricia E. Nicely. 1955. An analysis of perceptual confusions among some english consonants. The Journal of the Acoustical Society of America, 27(2):338–352. find
  23. Charlotte Pouw, Marianne de Heer Kloots, Afra Alishahi, and Willem Zuidema. 2024. Perception of phonological assimilation by neural speech recognition models. Computational Linguistics, 50(4):1557–1585. find
  24. Arne Rubehn, Jessica Nieder, Robert Forkel, and Johann-Mattis List. 2024. Generating feature vectors from phonetic transcriptions in cross-linguistic data formats. In Proceedings of the Society for Computation in Linguistics 2024, pages 205–216, Irvine, CA. Association for Computational Linguistics. find
  25. Andras Vargha and Harold D. Delaney. 2000. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25(2):101–132. find
  26. Qiantong Xu, Alexei Baevski, and Michael Auli. 2022. Simple and effective zero-shot cross-lingual phoneme recognition. In Interspeech 2022, pages 2113–2117. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/stafford-2026-what-do-neural-speech-models/