Elamur Research⌕
☰
home / papers / 2026 / Phonetic-based Ranking for Improved Pseudo-Labeling in Low-Resource ASR

Phonetic-based Ranking for Improved Pseudo-Labeling in Low-Resource ASR

Summary

Pseudo-labeling means fine-tuning an ASR model on audio that another system has transcribed automatically. It works when the transcriber is reasonably accurate. For poorly supported languages, however, error rates can exceed 50%, and existing filters that assume pseudo-labels with roughly 20–25% WER become unreliable. The FBK authors propose a cheap, language-independent way to score each pseudo-label. It needs no extra training and no access to ASR confidence scores.

Each utterance is processed in two ways. A fine-tuned whisper-large-v3 produces a word transcript, which an open-source phonemizer converts to IPA. Separately, wav2vec2-lv-60-espeak-cv-ft recognizes IPA symbols directly from the audio. The edit distance between the two phone strings gives a per-utterance PER, which is used to rank the data. The best-ranked utterances are then added when fine-tuning the source Whisper models.

On Lithuanian, Maltese and Slovenian benchmarks, this selection lowers WER by about 2–3 points. The phonetic recognizer draws on cross-lingual acoustic knowledge at low cost, so filtering remains feasible even when the initial ASR is weak.

Key points

  • Pipeline: whisper-large-v3 (W) transcripts are phonemized with the bootphon phonemizer (P) and compared via edit distance to IPA output from wav2vec2-lv-60-espeak-cv-ft (E). The resulting PER serves as the quality estimate.
  • Validation (Table 1): PER was compared against real CER on the Vox Populi test sets (lt, sl) and the MASRI test set (mt). Pearson correlations range from about 0.86 to 0.98.
  • Comparison with a second ASR: these correlations are comparable to, and on average slightly better than, an estimator built from two ASR systems (Whisper vs. mms-1b-fl102). The phonetic model is also much faster.
  • Selection strategies: utterances are kept either below a PER threshold, trading quantity against quality, or as the top-K of the ranking.
  • Advantages and results: the ASR is treated as a black box, no preliminary training is needed, and the method is language-agnostic. Fine-tuning on the filtered pseudo-labels gives up to a 3-point WER reduction on lt, mt and sl.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/338dnb8n7e85

@inproceedings{matassoni2026phonetic,
  title = {{Phonetic-based Ranking for Improved Pseudo-Labeling in Low-Resource ASR}},
  author = {Matassoni, Marco and Gretter, Roberto and Daniele, Falavigna and Nawar, Mohamed Nabih Ali Mohamed and Brutti, Alessio and Negri, Matteo and Cettolo, Mauro and Gaido, Marco and Papi, Sara and Bentivogli, Luisa},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/338dnb8n7e85},
  url = {https://doi.org/10.63317/338dnb8n7e85}
}

References

52 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Ali and Steve Renals. 2018. Word error rate estimation for speech recognition: e-WER. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 20–24. find
  2. Ahmed Ali and Steve Renals. 2020. Word error rate estimation without ASR output: e-WER2. In Interspeech 2020, pages 616–620. find
  3. R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber. 2020. Common voice: A massively-multilingual speech corpus. In 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4211–4215. find our card
  4. Ahmed Attia, Dorottya Demszky, Jing Liu, and Carol Espy-Wilson. 2025. From Weak Labels to Strong Results: Utilizing 5,000 Hours of Noisy Classroom Transcripts with Minimal Accurate Data. In Interspeech 2025, pages 3678–3682. find
  5. Martijn Bartelds, Nay San, Bradley McDonnell, Dan Jurafsky, and Martijn B. Wieling. 2023. Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation. In Annual Meeting of the Association for Computational Linguistics. find
  6. Mathieu Bernard and Hadrien Titeux. 2021. Phonemizer: Text to phones transcription for multiple languages in python. Journal of Open Source Software, 6(68):3958. find
  7. Kaushal Santosh Bhogale, Deovrat Mehendale, Niharika Parasa, Sathish Kumar Reddy G, Tahir Javed, Pratyush Kumar, and Mitesh M. Khapra. 2024. Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling. In Interspeech, pages 2519–2523. find
  8. Yao-Fei Cheng, Li-Wei Chen, Hung-Shin Lee, and Hsin-min Wang. 2024. Exploring the impact of data quantity on asr in extremely low-resource languages. find
  9. Shammur Absar Chowdhury and Ahmed Ali. 2023. Multilingual word error rate estimation: e-WER3. In ICASSP, pages 1–5. IEEE. find
  10. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2023. FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 798–805. find our card
  11. Dimitrios Damianos, Georgios Paraskevopoulos, and Alexandros Potamianos. 2025. MSDA: Combining Pseudo-labeling and Self-Supervision for Unsupervised Domain Adaptation in ASR. In Interspeech 2025, pages 3863–3867. find
  12. Linus Ericsson, Henry Gouk, Chen Change Loy, and Timothy M. Hospedales. 2022. Self-Supervised Representation Learning: Introduction, advances, and challenges. IEEE Signal Processing Magazine, 39(3):42–62. find
  13. Muhammad Umar Farooq, Rehan Ahmad, and Thomas Hain. 2023. MUST: A Multilingual Student-Teacher Learning Approach for Low-Resource Speech Recognition. 2023 IEEE Automatic Speech Recognition and Understanding Workshop, pages 1–6. find
  14. Jiayi Fu and Kuang Ru. 2019. A dropout-based single model committee approach for active learning in asr. In Proc. of IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 16–22. find
  15. Marco Gaido, Sara Papi, Luisa Bentivogli, Alessio Brutti, Mauro Cettolo, Roberto Gretter, Marco Matassoni, Mohamed Nabih, and Matteo Negri. 2024. MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages. In 2024 Conference on Empirical Methods in Natural L… find
  16. Dongji Gao, Hainan Xu, Desh Raj, Leibny Paola García-Perera, Daniel Povey, and Sanjeev Khudanpur. 2023. Learning From Flawed Data: Weakly Supervised Automatic Speech Recognition. In IEEE Automatic Speech Recognition and Understanding Workshop, pages 1–8. find
  17. Carlos Daniel Hernandez Mena. 2023. Acoustic Model in Maltese: whisper-large-maltese-8ksteps-64h. find
  18. Carlos Daniel Hernandez Mena, Ayrton-Didier Brincat, Albert Gatt, Andrea DeMarco, Claudia Borg, Lonneke van der Plas, and Iván Vladimir Meza Ruiz. 2020. MASRI-TEST CORPUS: Audio and Transcriptions in Maltese extracted from the YouTube channel of the University of Malta. find
  19. Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, and Takaaki Hori. 2022. Momentum pseudo-labeling: Semi-supervised asr with continuously improving pseudo-labels. IEEE Journal of Selected Topics in Signal Processing, 16(6):1424– 1438. find
  20. Minu Kim, Kangwook Jang, and Hoirin Kim. 2025. Improving cross-lingual phonetic representation of low-resource languages through language similarity analysis. In IEEE International Conference on Acoustics, Speech and Signal Processing. find
  21. Léa-Marie Lam-Yee-Mui, Waad Ben Kheder, Viet Bac Le, Claude Barras, and Jean-Luc Gauvain. 2023. Multilingual Models with Language Embeddings for Low-resource Speech Recognition. 2nd Annual Meeting of the ELRA/ISCA SIG on Under-resourced Languages (SIGUL). find
  22. Jaeyoung Lee, Masato Mimura, and Tatsuya Kawahara. 2025. Leveraging ipa and articulatory features as effective inductive biases for multilingual asr training. find
  23. ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). find
  24. Tatiana Likhomanenko, Ronan Collobert, Navdeep Jaitly, and Samy Bengio. 2023. Continuous soft pseudo-labeling in asr. volume 187 of Proceedings of Machine Learning Research, pages 66– 84. PMLR. find
  25. Shaoshi Ling, Chen Shen, Meng Cai, and Zejun Ma. 2022. Improving pseudo-label training for end-to-end speech recognition using gradient mask. In ICASSP, pages 8397–8401. IEEE. find
  26. Chunxi Liu, Qiaochu Zhang, Xiaohui Zhang, Kritika Singh, Yatharth Saraf, and Geoffrey Zweig. 2020. Multilingual graphemic hybrid ASR with massive data augmentation. In Proceedings of SLTU and CCURL, pages 46–52, Marseille, France. find
  27. Y. Long, M. J. F. Gales, P. Lanchantin, X. Liu, M. S. Seigel, and P. C. Woodland. 2013. Improving lightly supervised training for broadcast transcription. In Proc. of Interspeech, pages 2187–2191. find
  28. Hongxuan Lu and Biao Li. 2024. Sample adaptive data augmentation with progressive scheduling. find
  29. Toneva Mariya, Sordoni Alessandro, Tachet des Combes Remi, Trischler Adam, Bengio Yoshua, and Geoffrey J. Gordon. 2019. An empirical study of example forgetting during deep neural network learning. In Proc. of ICLR. find
  30. Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. 2021. Deep learning on a data diet: Finding important examples early in training. In Proc. of Neurips, volume 34. find
  31. Andrés Piñeiro-Martín, Carmen García-Mateo, Laura Docío-Fernández, Maria del Carmen Lopez-Perez, and Georg Rehm. 2024. Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition. Interspeech. find
  32. Rohit Prabhavalkar, Takaaki Hori, Tara N. Sainath, Ralf Schluter, and Shinji Watanabe. 2023. End-to-End Speech Recognition: A Survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:325–351. find
  33. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In 40th International Conference on Machine Learning. find our card
  34. Pradeep Rangappa, Andrés Carofilis, Jeena Prakash, Shashi Kumar, Sergio Burdisso, Srikanth Madikeri, Esaú Villatoro-Tello, Bidisha Sharma, Petr Motlicek, Kadri Hacioglu, Shankar Venkatesan, Saurabh Vyas, and Andreas Stolcke. 2025a. Efficient Data Selection for Domain Adaptation of ASR Using Pseudo-L… find
  35. Pradeep Rangappa, Juan Zuluaga-Gomez, Srikanth Madikeri, Andres Carofilis, Jeena Prakash, Sergio Burdisso, Shashi Kumar, Esaú Villatoro-Tello, Iuliia Nigmatulina, Petr Motlicek, Karthik Pandia, and Aravind Ganapathiraju. 2025b. Speech data selection for efficient asr fine-tuning using domain classif… find
  36. Nithin Rao Koluguri, Monica Sekoyan, George Zelenfroynd, Sasha Meister, Shuoyang Ding, Sofia Kostandian, He Huang, Nikolay Karpov, Jagadeesh Balam, Vitaly Lavrukhin, Yifan Peng, Sara Papi, Marco Gaido, Alessio Brutti, and Boris Ginsburg. 2025. Granary: Speech Recognition and Translation Dataset in 2… find
  37. Giuseppe Riccardi and Dilek Hakkani-Tur. 2005. Active learning: Theory and applications to automatic speech recognition. Speech and Audio Processing, IEEE Transactions on, 13:504 – 511. find
  38. Nay San, Georgios Paraskevopoulos, Aryaman Arora, Xiluo He, Prabhjot Kaur, Oliver Adams, and Dan Jurafsky. 2024. Predicting positive transfer for improved low-resource speech recognition using acoustic pseudo-tokens. In Proceedings of the 6th Workshop on Research in Computational Linguistic Typology… find
  39. Tanja Schultz and Tim Schlippe. 2014. GlobalPhone: Pronunciation dictionaries in 20 languages. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC‘14), pages 337–341. find
  40. Mingchen Shao, Xinfa Zhu, Chengyou Wang, Bingshen Mu, Hai Li, Ying Yan, Junhui Liu, Danming Xie, and Lei Xie. 2025. Weakly Supervised Data Refinement and Flexible Sequence Compression for Efficient Thai LLM-based ASR. In Interspeech 2025, pages 748–752. find
  41. Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. 2022. Beyond neural scaling laws: beating power law scaling via data pruning. In Proc. of Neurips, volume 35, pages 19523–19536. find our card
  42. Chihiro Taguchi, Yusuke Sakai, Parisa Haghani, and David Chiang. 2023. Universal Automatic Phonetic Transcription into the International Phonetic Alphabet. In Interspeech 2023, pages 2548– 2552. find
  43. Raphael Tang, Karun Kumar, Gefei Yang, Akshat Pandey, Yajie Mao, Vladislav Belyaev, Madhuri Emmadi, Craig Murray, Ferhan Ture, and Jimmy Lin. 2022. SpeechNet: Weakly supervised, end-to-end speech recognition at industrial scale. In Proceedings of the 2022 Conference on Empirical Methods in Natural L… find
  44. Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021. VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation. pages 993–1003. Association for Co… find our card
  45. Transformers: State-ofthe-Art Natural Language Processing. In 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45. Association for Computational Linguistics. find
  46. Qiantong Xu, Alexei Baevski, and Michael Auli. 2022. find
  47. Simple and Effective Zero-shot Cross-lingual Phoneme Recognition. In Interspeech 2022, pages 2113–2117. find
  48. Qiantong Xu, Tatiana Likhomanenko, Jacob Kahn, Awni Hannun, Gabriel Synnaeve, and Ronan Collobert. 2020. Iterative pseudo-labeling for speech recognition. In Interspeech 2020, pages 1006–1010. find
  49. Jianwei Yu, Hangting Chen, Yanyao Bian, Xiang Li, Yimin Luo, Jinchuan Tian, Mengyang Liu, Jiayi Jiang, and Shuai Wang. 2024. AutoPrep: An Automatic Preprocessing Framework for In-The-Wild Speech Data. 2024 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1136–1140. find
  50. Zhongzhi Yu, Yang Zhang, Kaizhi Qian, Cheng Wan, Yonggan Fu, Yongan Zhang, and Yingyan (Celine) Lin. 2023. Master-asr: achieving multilingual scalability and low-resource adaptation in asr with modular learning. In Proceedings of the 40th International Conference on Machine Learning. find
  51. Saierdaer Yusuyin, Te Ma, Hao Huang, Wenbo Zhao, and Zhijian Ou. 2024. Whistle: Data-efficient multilingual and crosslingual speech recognition via weakly phonetic supervision. IEEE Transactions on Audio, Speech and Language Processing, 33:1440–1453. find
  52. Rui Zhou, Takaki Koshikawa, Akinori Ito, Takashi Nose, and Chia-Ping Chen. 2024. Multilingual Meta-Transfer Learning for Low-Resource Speech Recognition. IEEE Access, 12:158493– 158504. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/matassoni-2026-phonetic-based-ranking-improved-pseudo/