Elamur Research⌕
☰
home / papers / 2026 / Synthetic Doctor-Patient Dialogue Generation for Robust Medical ASR: A Scalable Pipeline for Vocabulary Expansion and Privacy Preservation

Synthetic Doctor-Patient Dialogue Generation for Robust Medical ASR: A Scalable Pipeline for Vocabulary Expansion and Privacy Preservation

Paper · 2026 · Kefei Liu and Meizhu Liu

Summary

General-purpose speech recognizers such as Whisper make many errors on clinical speech, particularly on drug names. Real annotated medical audio is scarce because of privacy rules and the cost of labeling. Liu and Liu address this with a fully synthetic data pipeline that needs no real patient recordings.

They first assembled a lexicon of about 124,000 medical terms. Five open-source LLMs then wrote doctor–patient conversations around each term, guided by structured prompts that separate clinician and lay-patient speech styles, control where terms appear, and randomize personas. A two-stage filter removed weak dialogues: rule-based checks first, then GPT-4 scoring. The surviving dialogues were voiced using voice cloning with accent adaptation, plus injected noise and clinic-like sound effects.

Fine-tuning ASR models on this corpus lowered overall WER and improved recognition of medical terms. The work shows a scalable, privacy-safe way to adapt ASR to specialized domains with fast-changing vocabulary. The method could transfer to other low-resource settings.

Key points

  • Lexicon: over 24,000 medication names were drawn from public drug lists (Beck 2023, Drugs.com). Another 100,000 clinical concepts (diseases, symptoms, procedures, anatomy) came from the UMLS Metathesaurus.
  • Generators: the LLM ensemble was Llama3.1-8B, Mixtral-8x7B-Instruct-v0.1, GPT-2 XL, Flan-T5 and Falcon-7B-Instruct. Each model produced 100 dialogues per term, giving about 620 million text conversations and over 1 billion audio clips with ground-truth transcripts.
  • Prompting: eight prompting techniques were used. These include role-specific instructions, turn-by-turn scaffolding, controlled term injection, case anchors, hesitation and style cues, safety constraints, persona variation and self-consistency checks.
  • Remaining errors: the authors' error analysis found persistent problems with rare, hard-to-pronounce drugs (e.g., talimogene laherparepvec) and with homophones (e.g., ileum/ilium, mucus/mucous). Low-SNR noise and overlapping speech also hurt accuracy.
  • Limitations: the data is English-only and has no multi-speaker scenarios. Evaluation used only open-source ASR models, and coverage of newly emerging terms is necessarily incomplete.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Science and Technology of ChinaHefei · China145 works · 114,409 citations🎓University of FloridaGainesville · United States36 works · 364 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSignal Processing · 493 works

🧭 Topics 2Research subjects and languages the work is about.Dialogue systems · 654 worksSpeech & spoken language · 519 works

Cite

Cite as Kefei Liu and Meizhu Liu. 2026. Synthetic Doctor-Patient Dialogue Generation for Robust Medical ASR: A Scalable Pipeline for Vocabulary Expansion and Privacy Preservation. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-industry.40

@inproceedings{liu2026synthetic,
  title = {{Synthetic Doctor-Patient Dialogue Generation for Robust Medical ASR: A Scalable Pipeline for Vocabulary Expansion and Privacy Preservation}},
  author = {Kefei Liu and Meizhu Liu},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.40},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.40}
}

References

7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hussam Azzuni and Abdulmotaleb El Saddik. 2024. Voice cloning: Comprehensive survey. https://arxiv.org/html/2505.00579v1. ↗
  2. Eunbeen Jo and et a. 2024. Language models are unsupervised multitask learners. Journal of Medical Internet Research. find
  3. Hussam Azzuni and Abdulmotaleb El Saddik. 2025. Voice cloning: Comprehensive survey. arXiv preprint. find
  4. Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, and Graham Neubig. 2024. Evaluating language models as synthetic data generators.
  5. Sourav Banerjee, Ayushi Agarwal, and Promila Ghosh. 2024. High-precision medical speech recognition through synthetic data and semantic correction: United-medasr. find
  6. Feng-Ting Liao, Yung-Chieh Chan, Yi-Chang Chen, Chan-Jan Hsu, and Da shan Shiu. 2024. Zero-shot domain-sensitive speech recognition with prompt-conditioning fine-tuning. arXiv preprint. find
  7. Hali Lindsay, Johannes Tröger, Mario Mina, Nicklas Linz, Philipp Müller, Jan Alexandersson, and Inez Ramakers. 2022. Generating synthetic clinical speech data through simulated asr deletion error. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2026-synthetic-doctor-patient-dialogue-generation/