Elamur Research⌕
☰
home / papers / 2025 / Mispronunciation Detection Without L2 Pronunciation Dataset in Low-Resource Setting: A Case Study in Finland Swedish

Mispronunciation Detection Without L2 Pronunciation Dataset in Low-Resource Setting: A Case Study in Finland Swedish

Paper · 2025 · Nhan Phan, Mikko Kuronen, Maria Kautonen et al.

Summary

The paper tackles mispronunciation detection (MD) for Finland Swedish (FS), a minority variety of Swedish that has no public corpus of learner speech annotated for pronunciation. The authors avoid needing such a corpus. They train only on native (L1) spontaneous speech, about 89 hours drawn from the Talko3 interview corpus and the Aalto FS Parliament ASR corpus. The model is the multilingual XLS-R wav2vec 2.0, fine-tuned on orthographic rather than phonetic transcripts and trained with entropy regularization. After inference, temperature scaling and top-k normalization reshape its output probabilities so they work better for flagging errors. For testing, they reuse 33 minutes of L2 read-aloud speech from DigiTala. There, mismatches between the transcriber's verbatim transcript and the target text serve as error labels. The method shifts the trade-off toward precision: 29.8% precision at 43.2% recall, against 17.6% precision at 77.5% recall for the baseline. This matters because the pipeline is simple, needs almost no learner data, and should carry over to other low-resource languages and varieties.

Key points

  • Training data comes from two corpora: 83 hours of Talko3, kept to clips of 2–25 s and split by speaker (68 h/388 speakers for training, 15 h/80 for development), and about 6 hours of the Aalto FS Parliament ASR corpus. No Sweden Swedish data was used.
  • The test set is 485 DigiTala read-aloud samples, a mix of short words and sentences totalling about 33 minutes. DigiTala's spontaneous portion was deliberately excluded from training so the model would not overfit to one transcriber's style.
  • The model is XLS-R with 300M parameters, pre-trained on 436,000 hours of speech in 128 languages. It is trained with entropy regularization, and temperature scaling plus top-k normalization are applied after inference.
  • Orthographic targets were chosen for two reasons. Phonetic annotation is very costly (Talko3's took over 10 years and 20 people), and orthography lets several valid FS pronunciations count as correct, such as a short or long first vowel in fara or göra.
  • Three design principles guide the work: a standard-variety target, intelligibility rather than native-likeness, and inclusion of all dialects. For pedagogical reasons these lead the authors to favour precision over recall.

Interesting ideas

  • I1. When no gold error labels exist, verbatim transcripts can be compared against known target text to recover weak error labels. Such a test set is then used only to rank models against each other, never as an absolute score.
  • I2. Entropy regularization during training, combined with post-hoc temperature scaling, makes the model's confidence usable as a flagging signal instead of an over-sharp argmax.
  • I3. Deliberately preferring precision means users only see a few errors that are very likely real, since false alarms do more harm than misses.
  • I4. Using a target representation coarse enough to accept several legitimate surface variants avoids penalizing valid dialectal choices.
  • I5. Training a detector only on "correct" native data and treating low-confidence matches as anomalies removes the need for annotated error data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova could build cheap noisy evaluation sets from texts that have a known "intended" form and use them strictly to compare system versions. Negative controls would check that the ranking is not an artefact of the noise.
  • I2 (tree): Calibrating the perceptron and gparse arc scores the same way would give reliable thresholds for deciding when a parse is shaky enough to send to an LLM teacher tier or to leave out of silver data.
  • I3 (answer): When Mova grades school-course test answers or explains a mistake, it could report only high-confidence faults and stay silent otherwise, which protects trust in each explanation.
  • I4 (tokens): The lemmatizer and morphology checks could accept a set of acceptable spellings or forms (e.g., British/American variants) as equally correct rather than forcing one canonical form.
  • I5 (tree): A model fit only on clean treebank trees could flag improbable arcs in new parses as likely errors, feeding candidates to the self-induced repair rules.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Low-resource languages · 267 works

Cite

Cite as Nhan Phan, Mikko Kuronen, Maria Kautonen, Riikka Ullakonoja, Anna von Zansen, Yaroslav Getman, Ekaterina Voskoboinik, Tamas Grosz and Mikko Kurimo. 2025. Mispronunciation Detection Without L2 Pronunciation Dataset in Low-Resource Setting: A Case Study in Finland Swedish. arXiv:2506.01156 · Proc. Interspeech 2025, pp. 2435-2439.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.21437/interspeech.2025-1375 · arXiv 2506.01156

@misc{phan2025mispronunciation,
  title = {{Mispronunciation Detection Without L2 Pronunciation Dataset in Low-Resource Setting: A Case Study in Finland Swedish}},
  author = {Nhan Phan, Mikko Kuronen, Maria Kautonen, Riikka Ullakonoja, Anna {von Zansen}, Yaroslav Getman, Ekaterina Voskoboinik, Tamas Grosz and Mikko Kurimo},
  year = {2025},
  doi = {10.21437/interspeech.2025-1375},
  eprint = {2506.01156},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.21437/interspeech.2025-1375}
}

References

29 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. [1] J. M. Levis, Intelligibility, oral communication, and the teaching of pronunciation. Cambridge University Press, 2018. find
  2. [2] M. Kautonen and M. Kuronen, Uttalsinlärning med fokus på svenska. Svenska litteratursällskapet i Finland, 2021. find
  3. [3] J.-O. Östman and L. Mattfolk, “Ideologies of standardisation: Finland Swedish and Swedish-language Finland,” Standard languages and language standards in a changing Europe, vol. 1, pp. 75–82, 2011. find
  4. [4] P. Helgason, C. Ringen, and K. Suomi, “Swedish quantity: Central Standard Swedish and Fenno-Swedish,” Journal of Phonetics, vol. 41, no. 6, pp. 534–545, 2013. ↗
  5. [14] A.-M. Ivars, Dialekter och småstadsspråk, ser. Svenskan i Finland – i dag och i går I:1. Skrifter utgivna av Svenska litter-atursällskapet i Finland (798), 2015. find
  6. [15] J. M. Levis, “Changing contexts and shifting paradigms in pronunciation teaching,” TESOL quarterly, vol. 39, no. 3, pp. 369– 377, 2005. ↗
  7. [16] L. F. Bachman, Fundamental Considerations in Language Testing. Oxford university press, 1990. find
  8. [17] M. Eskenazi, “An overview of spoken language technology for education,” Speech Communication, vol. 51, no. 10, pp. 832–844, Oct. 2009. ↗
  9. [18] R. Ellis, “Corrective feedback and teacher development,” L2 Journal: An electronic refereed journal for foreign and second language educators, vol. 1, no. 1, 2009. ↗
  10. [19] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020.
  11. [20] A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y. Saraf, J. Pino, A. Baevski, A. Conneau, and M. Auli, “XLS-R: Self-supervised cross-lingual speech representation learning at scale,” in Interspeech 2022, 2022, pp. 2278–2282.
  12. [21] L. Peng, K. Fu, B. Lin, D. Ke, and J. Zhan, “A study on fine-tuning wav2vec2.0 model for the task of mispronunciation detection and diagnosis,” in Interspeech 2021, 2021, pp. 4448–4452. find
  13. [22] L. Kürzinger, D. Winkelbauer, L. Li, T. Watzel, and G. Rigoll, “CTC-segmentation of large corpora for German end-to-end speech recognition,” in International Conference on Speech and Computer. Springer, 2020, pp. 267–278. find
  14. [5] M. Lounis, B. Dendani, and H. Bahi, “Mispronunciation detection and diagnosis using deep neural networks: A systematic review,” Multimedia Tools and Applications, vol. 83, no. 23, pp. 62 793– 62 827, Jul. 2024. ↗
  15. [23] A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 369–376. find
  16. [6] G. Zhao, S. Sonsaat, A. Silpachai, I. Lucic, E. Chukharev-Hudilainen, J. Levis, and R. Gutierrez-Osuna, “L2-ARCTIC: A non-native English speech corpus,” in Interspeech 2018. find
  17. [24] H. Liu, S. Jin, and C. Zhang, “Connectionist temporal classification with maximum entropy regularization,” Advances in Neural Information Processing Systems, vol. 31, 2018. find
  18. [7] J. Zhang, Z. Zhang, Y. Wang, Z. Yan, Q. Song, Y. Huang, K. Li, D. Povey, and Y. Wang, “speechocean762: An open-source non-native English speech corpus for pronunciation assessment,” in Interspeech 2021, 2021, pp. 3710–3714.
  19. [8] P. Langlais, A.-M. Öster, and B. Granström, “Phonetic-level mispronunciation detection in non-native Swedish speech,” in 5th International Conference on Spoken Language Processing (ICSLP 1998), 1998, p. paper 0311. find
  20. [9] A. O’Neil, N. Hjortnaes, F. Tyers, Z. Nkosi, T. Ndlovu, Z. Mlondo, and N. P. Pewa, “Developing a benchmark for pronunciation feedback: Creation of a phonemically annotated speech corpus of isiZulu language learner speech,” in Proceedings of LREC-COLING 2024. find
  21. ELRA and ICCL, May 2024, pp. 4795– 4801. find
  22. [10] A. Lee and J. Glass, “Mispronunciation detection without nonnative training data,” in Interspeech 2015, 2015, pp. 643–647. find
  23. [11] N. Phan, T. Grósz, and M. Kurimo, “CaptainA - a mobile app for practising Finnish pronunciation,” in Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa). University of Tartu Library, May 2023, pp. 265–270. find
  24. [12] O.-V. Raitolahti, “Finland Swedish automatic speech recognition,” Master’s thesis, Aalto University, 2022. find
  25. [13] R. Al-Ghezi, K. Voskoboinik, Y. Getman, A. Von Zansen, H. Kallio, M. Kurimo, A. Huhta, and R. Hildén, “Automatic speaking assessment of spontaneous L2 Finnish and Swedish,” Language Assessment Quarterly, vol. 20, no. 4-5, pp. 421–444, 2023. ↗
  26. [25] X. Xu, Y. Kang, S. Cao, B. Lin, and L. Ma, “Explore wav2vec 2.0 for mispronunciation detection,” in Interspeech 2021, 2021, pp. 4428–4432. find
  27. [26] S. M. Witt and S. J. Young, “Phone-level pronunciation scoring and assessment for interactive language learning,” Speech communication, vol. 30, no. 2-3, pp. 95–108, 2000. ↗
  28. [27] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in International conference on machine learning. PMLR, 2017, pp. 1321–1330. ↗
  29. [28] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), 2020, pp. 4211–4215. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/phan-2025-mispronunciation-detection-l2-pronunciation-dataset/