Elamur Research⌕
☰
home / papers / 2026 / No Label? No Problem: Unsupervised Continual Learning for Adaptive Medical ASR

No Label? No Problem: Unsupervised Continual Learning for Adaptive Medical ASR

Paper · 2026 · Meizhu Liu and Tao Sheng

Summary

Medical speech recognition often fails on drug names, disease terms and other specialist vocabulary. The difficulty grows with accents, noisy clinical rooms and a steady stream of new terminology. Labeled clinical audio is expensive, needs expert annotators and is limited by privacy rules. The authors present MeSR, an online continual learning framework that needs no labels and builds on Whisper-large-v2.

The pipeline works in four stages:

  • The model transcribes incoming audio and assigns each output a confidence score (the mean softmax probability over its tokens).
  • A strict confidence cutoff keeps only very reliable pseudo-labels.
  • The kept clips are expanded with added noise, voice cloning and clinical sound effects.
  • The model is updated in a single pass with an Elastic Weight Consolidation loss scaled by confidence, plus selective fine-tuning of layers.

No raw audio is retained. On real medical recordings, both WER and EWER fall clearly compared with Whisper, and the gains also hold on general-domain benchmarks. The approach matters because it shows how a deployed medical ASR system can keep improving cheaply without manual annotation while respecting privacy.

Key points

  • Confidence threshold: It was calibrated on 22k labeled medical clips. The rule was that clips above the cutoff must have WER below 0.001 in at least 95% of cases.
  • Augmentation: Gaussian noise at several deviation levels, hospital equipment sounds, voice cloning from colleagues with varied accents, and more than 400 medical sound effects.
  • Main medical results: Compared with Whisper, overall WER dropped from 3.13% to 2.11% and EWER from 41.10% to 27.32%. The final model was trained continually on 2.1 million medical audio samples.
  • Comparison with alternatives:
    • Full-model LoRA fine-tuning reached 2.93 WER in 3.2 h, versus 2.11 WER in 1.6 h for MeSR.
    • Iterative pseudo-labeling reached 2.75 WER but took longer (reported as 9.7).
  • Open-dataset WER (Whisper → MeSR):
    • LibriSpeech test-other: 4.96 → 2.13
    • Common Voice: 8.85 → 3.16
    • CORAAL: 16.23 → 6.35
  • Limitations: The system has run in production for eight months. Low-confidence audio is discarded, and the model can still be confidently wrong. Tuning λ requires some labeled data, and the production data is not public.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 480 works

Cite

Cite as Meizhu Liu and Tao Sheng. 2026. No Label? No Problem: Unsupervised Continual Learning for Adaptive Medical ASR. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-industry.24

@inproceedings{liu2026no,
  title = {{No Label? No Problem: Unsupervised Continual Learning for Adaptive Medical ASR}},
  author = {Meizhu Liu and Tao Sheng},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.24},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.24}
}

References

32 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Steven Vander Eeckt and Hugo Van hamme. 2023. Rehearsal-free online continual learning for automatic speech recognition. In Interspeech, pages 944– 948. find
  2. Steven Vander Eeckt and Hugo Van hamme. 2024. Unsupervised online continual learning for automatic speech recognition. find
  3. Shiyu Fan, Nurmemet Yolwas, Wen Li, and Jinting Zhang. 2023. Iterative pseudo-labeling methods for improving speech recognition. find
  4. Mohammed Fouda. 2024. How ai is mitigating look-alike, sound-alike medication errors. find
  5. Li Fu, Xiaoxiao Li, Libo Zi, Zhengchen Zhang, Youzheng Wu, Xiaodong He, and Bowen Zhou. 2021. Incremental learning for end-to-end automatic speech recognition. find
  6. A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang. 2020. Conformer: Convolution-augmented transformer for speech recognition. In Interspeech, page 5036–5040.
  7. W. Han, Z. Zhang, Y. Zhang, J. Yu, C.-C. Chiu, J. Qin, A. Gulati, R. Pang, and Y. Wu. 2020. Contextnet: Improving convolutional neural networks for automatic speech recognition with global context. In Interspeech, page 3610–3614. find
  8. B. Houston and K. Kirchhoff. 2020. Continual learning for multidialect acoustic models. In Interspeech. find
  9. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR).
  10. Nazmul Kazi, Matt Kuntz, Upulee Kanewala, and Indika Kahanda. Dataset for automated medical transcription. find
  11. Tejumade Afonja, Tobi Olatunji, Sewade Ogun, Naome A. Etori, Abraham Owodunni, and Moshood Yekini. 2024. Performant asr models for medical entities in accented speech. find
  12. Harsh Ahlawat, Naveen Aggarwal, and Deepti Gupta. 2025. Automatic speech recognition: A survey of deep learning techniques and approaches. find
  13. Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2…
  14. Heng-Jui Chang, Hung yi Lee, and Lin shan Lee. 2021. Towards lifelong learning of end-to-end asr. find
  15. Tyler Kendall and Charlie Farrington. 2022. Coraal -online resources for african american language. Accessed: 2025-09-30. find
  16. Samuel Kessler, Bethan Thomas, and Salah Karout. 2021. Continual-wav2vec2: an application of continual learning for self-supervised automatic speech recognition. find
  17. James Kirkpatrick, Razvan Pascanu, Joel Veness Neil Rabinowitz, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. find
  18. Tatiana Likhomanenko, Qiantong Xu, Jacob Kahn, Gabriel Synnaeve, and Ronan Collobert. 2021. slim-ipl: Language-model-free iterative pseudo-labeling. find
  19. Alexander Ly, Maarten Marsman, Josine Verhagen, Raoul Grasman, and Eric-Jan Wagenmakers. 2017. How ai is mitigating look-alike, sound-alike medication errors. find
  20. Anqi Mao, Mehryar Mohri, and Yutao Zhong. 2023. Cross-entropy loss functions: Theoretical analysis and applications. find
  21. M. Zeineldeen, A. Glushko, W. Michel, A. Zeyer, R. Schluter, and H. Neyy. 2021. Investigating methods to improve language model integration for attention-based encoder-decoder asr models. In Interspeech, page 2856–2860. find
  22. Han Zhu, Dongji Gao, Gaofeng Cheng, Daniel Povey, Pengyuan Zhang, and Yonghong Yan. 2023. Alternative pseudo-labeling for semi-supervised automatic speech recognition. find
  23. Qingliang Meng, Hao Wu, Wei Liang, Wei Xu, and Qing Zhao. 2025. Ilt-iterative lora training through focus-feedback-fix for multilingual speech recognition. find
  24. Josh Meyer, Lindy Rauchenstein, Joshua D. Eisenberg, and Nicholas Howell. 2020. Artie bias corpus: An open dataset for detecting demographic bias in speech applications. In Proceedings of the Twelfth Language Resources and Evaluation Conference (LREC 2020), pages 6462–6468.
  25. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An asr corpus based on public domain audio books. In ICASSP. find
  26. Jeena Prakash, Blessingh Kumar, Kadri Hacioglu, Bidisha Sharma, Sindhuja Gopalan, Malolan Chetlur, Shankar Venkatesan, and Andreas Stolcke. 2025. Better pseudo-labeling with multi-asr fusion and error correction by speechllm. find
  27. Zengyi Qin, Wenliang Zhao, Xumin Yu, and Xin Sun. 2023. Openvoice: Versatile instant voice cloning. find
  28. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In Proceedings of the 40 th International Conference on Machine Learning.
  29. Brooke Schmidt. 2010. Look-alike drug name errors. find
  30. Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, Yannis Agiomvrgiannakis, and Yonghui Wu. 2018. Natural tts synthesis by conditioning wavenet on mel spectrogram predictions. In ICASSP, pages 4779– 4783. find
  31. Mengqian Wang, Ilya Valmianski, Xavier Amatriain, and Anitha Kannan. 2022. Learning functional sections in medical conversations: iterative pseudo-labeling and human-in-the-loop approach. find
  32. Qiantong Xu, Tatiana Likhomanenko, Jacob Kahn, Awni Hannun, Gabriel Synnaeve, and Ronan Collobert. 2020. Iterative pseudo-labeling for speech recognition. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-no-label-no-problem-unsupervised/