Elamur Research⌕
☰
home / papers / 2026 / ViMedCSS: A Vietnamese Medical Code-Switching Speech Dataset & Benchmark

ViMedCSS: A Vietnamese Medical Code-Switching Speech Dataset & Benchmark

Summary

Vietnamese clinicians often drop English terms such as drug names, procedures and biomarkers into otherwise Vietnamese speech. ASR systems tend to fail on exactly these insertions, which can corrupt clinical records and teaching materials, and no open benchmark has targeted the problem. The authors built ViMedCSS, a 34.57-hour corpus of 16,576 utterances, each containing one or more English medical terms. They took code-switched entries from VinUniversity's Meddict lexicon and used them as search queries for Vietnamese medical YouTube videos. Gemini 2.5 Pro transcribed and screened the audio. The pipeline then filtered utterances by language and domain, matched detected terms to dictionary entries via Levenshtein distance, and checked alignments manually. A separate hard split holds rare or unseen terms. The benchmark compares zero-shot multilingual and Vietnamese ASR systems with several fine-tuning strategies on a Whisper-based Vietnamese backbone, and it scores overall accuracy separately from accuracy on code-switched spans. Vietnamese-tuned models handle the matrix language better, while multilingual pretraining helps with English insertions. Combining the two gives the best balance, which offers practical guidance for low-resource domain ASR.

Key points

  • Corpus construction: ViMedCSS has 34.57 h of audio in 16,576 utterances of 3–29 s, covering five medical topics. It is released on Hugging Face as tensorxt/ViMedCSS.
  • Term lexicon: The authors extracted 3,203 code-switched terms from the 64,232-entry Meddict English–Vietnamese medical dictionary.
  • Source audio: More than 13,000 YouTube videos were crawled, and over 700 h of audio were processed before filtering reduced it to roughly 34 h of valid data.
  • Quality check: Two annotators reviewed a stratified sample of 500 utterances (about 1 h), with inter-annotator agreement of Cohen's κ = 0.65.
  • Models and strategies:
    • Zero-shot baselines: Whisper, PhoWhisper, MMS, wav2vec2-base-vi and VietASR.
    • Fine-tuning approaches: contextual biasing, LID-guided adaptation, LoRA adapters, post-decoding normalization, and hybrids of these.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/58uwrquo3znb

@inproceedings{nguyen2026vimedcss,
  title = {{ViMedCSS: A Vietnamese Medical Code-Switching Speech Dataset \& Benchmark}},
  author = {Nguyen, Tung X. and Vo, Nhu and Nguyen, Giang-Son and Hoang, Duy Mai and Huynh, Chien Dinh and Unanue, Iñigo Jauregi and Piccardi, Massimo and Buntine, Wray and Le, Dung D.},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/58uwrquo3znb},
  url = {https://doi.org/10.63317/58uwrquo3znb}
}

References

34 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bobbi Aditya, Mahdin Rohmatillah, Liang-Hsuan Tai, and Jen-Tzung Chien. 2024. Attention-Guided Adaptation for Code-Switching Speech Recognition. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10256– 10260. find
  2. Maha Tufail Agro, Atharva Kulkarni, Karima Kadaoui, Zeerak Talat, and Hanan Aldarmaki. 2025. Code-switching in end-to-end automatic speech recognition: A systematic literature review. find
  3. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, volume 33, pages 12449–12460. Curran Associates, Inc. find our card
  4. Stacy S Chen. 2025. A "code-switching" model for healthcare communication. Healthcare Management Forum, 38(4):391–394. find
  5. The Chuong Chu, Vu Tuan Dat Pham, Trung Kien Dao, Ngoc Hoang Nguyen, and Steven Truong. 2025. AdaCS: Adaptive Normalization for Enhanced Code-Switching ASR. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. find
  6. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46. find
  7. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2023. FLEURS: FEW-Shot Learning Evaluation of Universal Representations of Speech. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 798–805. find our card
  8. Abdullah Ashraf Hamad, Doaa B Mustaffa, Asmaa Zakria Alnajjar, Raghad Amro, Mohammad Ghassab Deameh, Bassant Amin, and Ibraheem M Alkhawaldeh. 2025. Decolonizing medical education: a systematic review of educational language barriers in countries using foreign languages for instruction. BMC Medical… find
  9. Haoxiang Hou, Xun Gong, Wangyou Zhang, Wei Wang, and Yanmin Qian. 2025. Ranking and Selection of Bias Words for Contextual Bias Speech Recognition. In Interspeech 2025, pages 5183– 5187. find
  10. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. find
  11. Thanh-Thien Le, Linh The Nguyen, and Dat Quoc Nguyen. 2024. PhoWhisper: Automatic Speech Recognition for Vietnamese. In Proceedings of the ICLR 2024 Tiny Papers track. find
  12. Khai Le-Duc. 2024. VietMed: A Dataset and Benchmark for Automatic Speech Recognition of Vietnamese in the Medical Domain. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 17365–17370, Torino, Italia. E… find
  13. Khai Le-Duc, Phuc Phan, Tan-Hanh Pham, Bach Phan Tat, Minh-Huong Ngo, Thanh Nguyen-Tang, and Truong-Son Hy. 2025. MultiMed: Multilingual Medical Speech Recognition via Attention Encoder Decoder. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Ind… find
  14. Hieu-Thi Luong and Hai-Quan Vu. 2016. Vivos: Vietnamese Speech Corpus for ASR. find
  15. Dau-Cheng Lyu, Tien-Ping Tan, Eng Siong Chng, and Haizhou Li. 2010. SEAME: a Mandarin-English code-switching speech corpus in southeast asia. In Interspeech 2010, pages 1986– 1989. find
  16. Thai Binh Nguyen. 2021. Vietnamese end-to-end speech recognition using wav2vec 2.0. find
  17. Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, et al. 2024. Scaling speech technology to 1,000+ languages. Journal of Machine Learning Research, 25(97):1–52. find our card
  18. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning, ICML’23. JMLR.org. find our card
  19. Samer H Sharkiya. 2023. Quality communication can improve patient-centred health outcomes among older patients: a rapid review. BMC Health Services Research, 23(1):886. find
  20. Xian Shi, Qiangze Feng, and Lei Xie. 2020. find
  21. The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Results. find
  22. Zheshu Song, Jianheng Zhuo, Yifan Yang, Ziyang Ma, Shixiong Zhang, and Xie Chen. 2024. find
  23. LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR. In Interspeech 2024, pages 3934–3938. find
  24. Yui Sudo, Yusuke Fujita, Atsushi Kojima, Tomoya Mizumoto, and Lianbo Liu. 2025. OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabulary. In Interspeech 2025, pages 5188–5192. find
  25. Yui Sudo, Yosuke Fukumoto, Muhammad Shakeel, Yifan Peng, and Shinji Watanabe. 2024. find
  26. Contextualized Automatic Speech Recognition With Dynamic Vocabulary. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 78– 85. find
  27. Guangzhi Sun, Xianrui Zheng, Chao Zhang, and Philip C. Woodland. 2023. Can Contextual Biasing Remain Effective with Whisper and GPT-2? In Interspeech 2023, pages 1289–1293. find
  28. Enes Yavuz Ugan, Ngoc-Quan Pham, and Alexander Waibel. 2024. DECM: Evaluating Bilingual ASR Performance on a Code-switching/mixing Benchmark. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4468–4475,… find
  29. Nhu Vo, Dat Quoc Nguyen, Dung D. Le, Massimo Piccardi, and Wray Buntine. 2024. Improving Vietnamese-English medical machine translation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 8955–8962, Tori… find
  30. Brian Yan, Injy Hamed, Shuichiro Shimizu, Vasista Sai Lodagala, William Chen, Olga Iakovenko, Bashar Talafha, Amir Hussein, Alexander Polok, Kalvin Chang, Dominik Klement, Sara Althubaiti, Puyuan Peng, Matthew Wiesner, Thamar Solorio, Ahmed Ali, Sanjeev Khudanpur, and Shinji Watanabe. 2025. find
  31. CS-FLEURS: A Massively Multilingual and Code-Switched Speech Dataset. In Interspeech 2025, pages 743–747. find
  32. Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. 2024. Zipformer: A faster and better encoder for automatic speech recognition. In ICLR. find
  33. Jiahui Zhao, Hao Shi, Chenrui Cui, Tianrui Wang, Hexin Liu, Zhaoheng Ni, Lingxuan Ye, and Longbiao Wang. 2025. Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decoding. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (IC… find
  34. Jianheng Zhuo, Yifan Yang, Yiwen Shao, Yong Xu, Dong Yu, Kai Yu, and Xie Chen. 2025. VietASR: Achieving Industry-level Vietnamese ASR with 50-hour labeled data and Large-Scale Speech Pretraining. In Interspeech 2025, pages 1163– 1167. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2026-vimedcss-vietnamese-medical-code-switching/