Elamur Research⌕
☰
home / papers / 2026 / VoiceBench: Benchmarking LLM-Based Voice Assistants

VoiceBench: Benchmarking LLM-Based Voice Assistants

Paper · 2026 · Yiming Chen, Xianghu Yue, Chen Zhang et al.

Summary

Voice assistants built on audio-capable LLMs, such as GPT-4o-style systems, have had no common yardstick. Earlier work mostly reported ASR scores, anecdotal examples, or small spoken QA sets made from clean TTS audio. The authors address this with VoiceBench, an evaluation suite of 6,783 spoken instructions in eight tasks. Some instructions are synthesized and some are recorded from human speakers with various accents. The tasks test three abilities: general knowledge (including reasoning and multiple-choice QA), following instructions, and refusing unsafe requests.

The suite also tests robustness. It applies controlled perturbations of three kinds:

  • Speaker: age, accent and pitch, produced with TTS and voice cloning.
  • Acoustic environment: reverberation, noise, compression and clipping.
  • Content: LLM-generated grammar slips, mispronunciations and disfluencies.

Tests across many open assistants show that a simple ASR-plus-LLM pipeline still clearly beats end-to-end models and copes better with perturbations. The quality of the speech encoder largely determines robustness. The work matters because it shows that ASR accuracy alone poorly predicts assistant quality, so responses must be judged directly.

Key points

  • Data: 6,783 spoken instructions over eight tasks. Sources include BBH (Hyperbaton, Sports Understanding, Web of Lies, Navigate), OBQA, and MMSU. MMSU is built from MMLU-Pro, cut to four answer options, with math and computer science removed.
  • Prior evaluations were limited: for example, LLaMA-Omni was tested on only 199 synthetic AlpacaEval queries.
  • Hardest perturbations: unfamiliar accents, environmental distortions and mispronunciations caused the largest drops. The authors recommend a strong pretrained encoder such as Whisper.
  • WER is unreliable as a proxy: at roughly 50% WER, response quality varied by about 1.5 points.
  • Limitations and availability: only text outputs and single-turn interactions are evaluated, and the variations are mostly simulated. Code and data are at github.com/MatthewCYM/VoiceBench.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓National University of SingaporeSingapore · Singapore84 works · 32,204 citations🔬Institute for Infocomm ResearchSingapore · Singapore33 works · 14,373 citations🏛️Agency for Science, Technology and ResearchSingapore · Singapore19 works · 13,974 citations🎓Chinese University of Hong Kong, ShenzhenShenzhen · China17 works · 2,364 citations🎓Tianjin UniversityTianjin · China11 works · 825 citations🎓Shenzhen Research Institute of Big DataShenzhen · China11 works · 1,615 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksSignal Processing · 213 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 233 works

Cite

Cite as Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T. Tan and Haizhou Li. 2026. VoiceBench: Benchmarking LLM-Based Voice Assistants. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.628

@article{chen2026voicebench,
  title = {{VoiceBench: Benchmarking LLM-Based Voice Assistants}},
  author = {Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T. Tan and Haizhou Li},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.628},
  url = {https://doi.org/10.1162/tacl.a.628}
}

References

66 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdi… find
  2. Andrew Caines, Christian Bentz, Kate Knill, Marek Rei, and Paula Buttery. 2020. Grammatical error detection in transcriptions of spoken English. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2144–2162, Barcelona, Spain (Online). International Committee on Co… find
  3. Ronald Carter and Michael McCarthy. 1995. Grammar and the spoken language. Applied Linguistics, 16(2):141–158. https://doi.org/10.1093/applin/16.2.141 2024b. Unveiling the achilles’ heel of NLG evaluators: A unified adversarial framework driven by large language models. In Findings of the Associati… ↗
  4. Kai Chen, Yunhao Gou, Runhui Huang, Zhili Liu, Daxin Tan, Jing Xu, Chunwei Wang, Yi Zhu, Yihan Zeng, Kuo Yang, Dingdong Wang, Kun Xiang, Haoyuan Li, Haoli Bai, Jianhua Han, Xiaohui Li, Weike Jin, Nian Xie, Yu Zhang, James T. Kwok, Hengshuang Zhao, Xiaodan Liang, Dit-Yan Yeung, Xiao Chen, Zhenguo Li,… find
  5. Simin Chen, Yiming Chen, Zexin Li, Yifan Jiang, Zhongwei Wan, Yixin He, Dezhi Ran, Tianle Gu, Haizhou Li, Tao Xie, and Baishakhi Ray. 2025b. Recent advances in large langauge model benchmarks against data contamination: From static to dynamic evaluation. arXiv preprint arXiv:2502.17521. https://doi.… — [doi:10.18653/v1/2025](https://doi.org/10.18653/v1/2025) ↗
  6. Yiming Chen, Simin Chen, Zexin Li, Wei Yang, Cong Liu, Robby Tan, and Haizhou Li. 2023. Dynamic transformers provide a false sense of efficiency. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7164–7180, Toronto, Canada. Asso… find
  7. Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, Ji Ma, Jiaqi Wang, Xiaoyi Dong, Hang Yan, Hewei Guo, Conghui He, Botian Shi, Zhenjiang Jin, Chao Xu, Bin Wang, Xingjian Wei, Wei Li, Wenjian Zhang, Bo Zhang, Pinlong Cai, Licheng W…
  8. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. Qwen2-audio technical report. arXiv preprint arXiv:2407.10759.
  9. Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919.
  10. Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics, 8:454–470…
  11. Yiming Chen, Xianghu Yue, Xiaoxue Gao, Chen Zhang, Luis Fernando D’Haro, Robby T. Tan, and Haizhou Li. 2024a. Beyond single-audio: Advancing multi-audio processing in audio large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10917–10930, Miami, Flor… find
  12. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: A speech-text foundation model for real-time dialogue. arXiv preprint arXiv: 2410.00037.
  13. Yiming Chen, Chen Zhang, Danqing Luo, Luis Fernando D’Haro, Robby Tan, and Haizhou Li. find
  14. Gary S. Dell and Peter A. Reich. 1981. Stages in sentence production: An analysis of speech error data. Journal of Verbal Learning and Verbal Behavior, 20(6):611–629. https://doi.org/10.1016/S0022-5371(81)90202-4 you desire. In Proceedings of the 2024 Conference of the North American Chapter of the… — [doi:10.1016/s0022-5371(81)90202-4](https://doi.org/10.1016/s0022-5371%2881%2990202-4) ↗
  15. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao, and Zhijie Yan. 2024. CosyVoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407. find
  16. Yassine El Kheir, Ahmed Ali, and Shammur Absar Chowdhury. 2023. Automatic pronunciation assessment - a review. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8304–8324, Singapore. Association for Computational Linguistics. https://doi.org/10.18653/v1/2023 .findings-e… ↗
  17. Fahim Faisal, Sharlina Keshava, Md Mahfuz Ibn Alam, and Antonios Anastasopoulos. 2021. SD-QA: Spoken dialectal question answering for the real world. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3296–3315, Punta Cana, Dominican Republic. Association for Computation… find
  18. Xiaoxue Gao, Yiming Chen, Xianghu Yue, Yu Tsao, and Nancy F. Chen. 2025a. TTslow: Slow down text-to-speech with efficiency robustness evaluations. IEEE Transactions on Audio, Speech and Language Processing, 33:693–704. https://doi.org/10.1109/TASLPRO .2025.3533357 — [doi:10.1109/taslpro](https://doi.org/10.1109/taslpro) ↗
  19. Xiaoxue Gao, Zexin Li, Yiming Chen, Cong Liu, and Haizhou Li. 2024. Transferable adversarial attacks against ASR. IEEE Signal Processing Letters, 31:2200–2204. https://doi.org/10.1109/LSP.2024.3443711 — [doi:10.1109/lsp.2024.3443711](https://doi.org/10.1109/lsp.2024.3443711) ↗
  20. Xiaoxue Gao, Chen Zhang, Yiming Chen, Huayun Zhang, and Nancy F. Chen. 2025b. EMO-DPO: Controllable emotional speech synthesis through direct preference optimization. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. https://doi.o… — [doi:10.1109/icassp49660](https://doi.org/10.1109/icassp49660) ↗
  21. Qingkai Fang, Shoutao Guo, Yan Zhou, Zhengrui Ma, Shaolei Zhang, and Yang Feng. 2025. LLaMA-Omni: Seamless speech interaction with large language models. In the Thirteenth International Conference on Learning Representations. find
  22. Xiaoxue Gao, Huayun Zhang, and Nancy F. Chen. 2025c. MultiGen: Child-friendly multilingual speech generator with LLMs. arXiv preprint arXiv:2508.08715. — [arXiv:2508.08715](https://arxiv.org/abs/2508.08715) find
  23. Jean E. Fox Tree. 1995. The effects of false starts and repetitions on the processing of subsequent words in spontaneous speech. Journal of Memory and Language, 34(6):709–738. https://doi.org/10.1006/jmla.1995 .1032 — [doi:10.1006/jmla.1995](https://doi.org/10.1006/jmla.1995) ↗
  24. Xiaoxue Gao, Huayun Zhang, and Nancy F. Chen. 2025d. Prompt-unseen-emotion: Zero-shot expressive speech synthesis with prompt-LLM contextual knowledge for mixed emotions. arXiv preprint arXiv:2506.02742. — [arXiv:2506.02742](https://arxiv.org/abs/2506.02742) find
  25. Yuan Gong, Hongyin Luo, Alexander H. Liu, Leonid Karlinsky, and James R. Glass. 2024. Listen, think, and understand. In the Twelfth International Conference on Learning Representations. find
  26. Chaoyou Fu, Haojia Lin, Zuwei Long, Yunhang Shen, Meng Zhao, Yifan Zhang, Xiong Wang, Di Yin, Long Ma, Xiawu Zheng, Rongrong Ji, Yunsheng Wu, Ran He, Caifeng Shan, and Xing Sun. 2024a. VITA: Towards open-source interactive omni multimodal LLM. arXiv preprint arXiv:2408.05211. — [arXiv:2408.05211](https://arxiv.org/abs/2408.05211) find
  27. Reinhold Haeb-Umbach, Shinji Watanabe, Tomohiro Nakatani, Michiel Bacchiani, Bjorn Hoffmeister, Michael L. Seltzer, Heiga Zen, and Mehrez Souden. 2019. Speech processing for Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2024b. GPTScore: Evaluate as digital home assistants: Combining sign… — [doi:10.1109/msp.2019](https://doi.org/10.1109/msp.2019) find
  28. William Held, Yanzhe Zhang, Minzhi Li, Weiyan Shi, Michael J. Ryan, and Diyi Yang. 2025. Distilling an end-to-end voice assistant without instruction training data. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7876–7891, Vi… find
  29. Chien-yu Huang, Ke-Han Lu, Shih-Heng Wang, Chi-Yuan Hsiao, Chun-Yi Kuan, Haibin Wu, Siddhant Arora, Kai-Wei Chang, Jiatong Shi, Yifan Peng, Roshan Sharma, Shinji Watanabe, Bhiksha Ramakrishnan, Shady Shehata, and Hung-yi Lee. 2024. Dynamic-Superb: Towards a dynamic, collaborative, and comprehensive… — [doi:10.1109/icassp48485](https://doi.org/10.1109/icassp48485) find
  30. Yidi Jiang, Qian Chen, Shengpeng Ji, Yu Xi, Wen Wang, Chong Zhang, Xianghu Yue, ShiLiang Zhang, and Haizhou Li. 2024. UniCodec: Unified audio codec with single domain-adaptive codebook. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 19112–19124. Ass… find
  31. Birger Kollmeier, Thomas Brand, and Bernd Meyer. 2008. Perception of speech and sound. Springer Handbook of Speech Processing, pages 61–82. https://doi.org/10.1007/978-3-540-49127-9 4 ↗
  32. Jean C. Krause and Louis D. Braida. 2004. Acoustic properties of naturally produced clear speech at normal speaking rates. Journal of the Acoustical Society of America, 115(1):362–378. https://doi.org/10.1121/1.1635842, PubMed: 14759028 ↗
  33. Jinyu Li, Li Deng, Yifan Gong, and Reinhold Haeb-Umbach. 2014. An overview of noise-robust automatic speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 22(4):745–777. https://doi.org/10.1109/TASLP.2014 .2304637 — [doi:10.1109/taslp.2014](https://doi.org/10.1109/taslp.2014) ↗
  34. Paria Jamshid Lou and Mark Johnson. 2020. End-to-end speech recognition and disfluency removal. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2051–2061, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.findings-emnlp.186 ↗
  35. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2024. BeaverTails: Towards improved safety alignment of LLM via a human-preference dataset. Advances in Neural Information Processing Systems.
  36. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. AlpacaEval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca eval Yadong Li, Haoze Sun, Mingan Lin, Tianpeng Li, Guosheng… ↗
  37. Zongxia Li, Ishani Mondal, Huy Nghiem, Yijun Liang, and Jordan Lee Boyd-Graber. 2024b. PEDANTS: Cheap but effective and interpretable answer equivalence. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9373–9398, Miami, Florida, USA. Association for Computational Ling… find
  38. Hongchao Jiang, Yiming Chen, Yushi Cao, Hung-yi Lee, and Robby T. Tan. 2025. CodeJudgeBench: Benchmarking LLM-as-a-judge for coding tasks. arXiv preprint arXiv: 2507.10535. https://doi.org/10.18653/v1/2025.acl-long.937 ↗
  39. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruction tuning.
  40. Advances in Neural Information Processing Systems, 36. find
  41. Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.…
  42. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. 2023. Trustworthy LLMs: A survey and guideline for evaluating large language models’ alignment. In Socially Responsible Language Modelling Research.
  43. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine…
  44. Benjamin Marie. 2023. Disfluency generation for more robust dialogue systems. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11479–11488, Toronto, Canada. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2023.findings-acl.728 find
  45. Elizabeth Ellen Shriberg. 1994. Preliminaries to a theory of speech disfluencies. Ph.D. thesis, Citeseer. find
  46. Michael McCarthy and Ronald Carter. 1995. Spoken grammar: What is it and how can we teach it? ELT Journal, 49(3):207–218. https://doi.org/10.1093/elt/49.3.207 — [doi:10.1093/elt/49.3.207](https://doi.org/10.1093/elt/49.3.207) ↗
  47. Julien Meyer, Laure Dentel, and Fanny Meunier. 2013. Speech recognition in natural background noise. PloS One, 8(11):e79279. https://doi.org/10.1371/journal.pone.0079279, PubMed: 24260183 — [doi:10.1371/journal.pone.0079279](https://doi.org/10.1371/journal.pone.0079279) ↗
  48. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? A new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391, Brussels, Belgium. Association fo… find
  49. Shuzheng Si, Wentao Ma, Haoyu Gao, Yuchuan Wu, Ting-En Lin, Yinpei Dai, Hangyu Li, Rui Yan, Fei Huang, and Yongbin Li. 2023. SpokenWOZ: A large-scale speech-text benchmark for spoken task-oriented dialogue agents. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Ben… find
  50. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. Challenging BIG-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics:…
  51. Aidar Myrzakhan, Sondos Mahmoud Bsharat, and Zhiqiang Shen. 2024. Open-LLM-Leaderboard: From multi-choice to open-style questions for LLMs evaluation, benchmark, and arena. arXiv preprint arXiv:2406.07545. — [arXiv:2406.07545](https://arxiv.org/abs/2406.07545) find
  52. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun MA, and Chao Zhang. 2024. SALMONN: Towards generic hearing abilities for large language models. In the Twelfth International Conference on Learning Representations.
  53. Qwen. 2025. Qwen2.5-Omni technical report. arXiv preprint arXiv:2503.20215.
  54. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492–28518. PMLR.
  55. Bin Wang, Chengwei Wei, Zhengyuan Liu, Geyu Lin, and Nancy F. Chen. 2024a. Resilience of large language models for noisy instructions. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11939–11950, Miami, Florida, USA. Association for Computational Linguistics. https://… find
  56. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. 2024b. MMLU-Pro: A more robust and challenging multi-task language understanding benchmar…
  57. Junkai Wu, Xulin Fan, Bo-Ru Lu, Xilin Jiang, Nima Mesgarani, Mark Hasegawa-Johnson, and Mari Ostendorf. 2024. Just ASR + LLM? A study on speech large language models’ ability to identify and understand speaker in spoken dialogue. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 1137–114… — [doi:10.1109/slt61566](https://doi.org/10.1109/slt61566) find
  58. Zhifei Xie and Changqiao Wu. 2024a. Mini-Omni: Language models can hear, talk while thinking in streaming. arXiv preprint arXiv: 2408.16725. https://doi.org/10.1109/TASLP.2017.2756440 ation for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long .303 ↗
  59. Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, and Jingren Zhou. 2024. AIR-Bench: Benchmarking large audio-language models via generative comprehension. In Proceedings of the 62nd Annual Meeting of the Association for Computati…
  60. Xianghu Yue, Xiaohai Tian, Malu Zhang, Zhizheng Wu, and Haizhou Li. 2025. COAVT: A cognition-inspired unified audio-visual-text pre-training model for multimodal processing. IEEE Transactions on Audio, Speech and Language Processing, 33:3255–3266. https://doi.org/10.1109/TASLPRO .2025.3587467 — [doi:10.1109/taslpro](https://doi.org/10.1109/taslpro) ↗
  61. Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2024. Evaluating large language models at evaluating instruction following. In the Twelfth International Conference on Learning Representations. find
  62. Chen Zhang, Luis Fernando D’Haro, Yiming Chen, Malu Zhang, and Haizhou Li. 2024a. A comprehensive analysis of the effectiveness of large language models as automatic dialogue evaluators. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Inn… find
  63. Zhifei Xie and Changqiao Wu. 2024b. Mini-Omni2: Towards open-source GPT-4o model with vision, speech and duplex. arXiv preprint arXiv:2410.11190. — [arXiv:2410.11190](https://arxiv.org/abs/2410.11190) find
  64. Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Michael L. Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig. 2017. Toward human parity in conversational speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(12):2410–2423. find
  65. Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. 2023. SpeechGPT: Empowering large language models with intrinsic cross-modal conversational abilities. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15757–15773, Singapore. Associa…
  66. Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. 2024. SafeDecoding: Defending against jailbreak attacks via safety-aware decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 55…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2026-voicebench-benchmarking-llm-based-voice/