Elamur Research⌕
☰
home / papers / 2025 / Step-Audio 2 Technical Report

Step-Audio 2 Technical Report

Paper · 2025 · Boyong Wu, Chao Yan, Chen Hu et al.

Summary

Step-Audio 2 is StepFun's end-to-end large audio language model (LALM). It targets three weaknesses of earlier systems: they ignore paralinguistic cues such as emotion and speaking style, they answer mostly in text, and they hallucinate because they lack external knowledge. The model reads raw audio through a latent audio encoder and produces interleaved text and discrete audio tokens. This lets it respond in speech whose tone matches the conversation. Chain-of-thought reasoning and reinforcement learning sharpen its understanding. Retrieval-augmented generation and tool calls ground its answers, including web search and a new audio search tool that retrieves voices so the model can change timbre on request. Training is multi-stage and uses 680 billion text tokens plus 8 million hours of real and synthetic audio. The authors report state-of-the-art results against open-source and commercial rivals on ASR, audio understanding, speech translation and spoken dialogue. This makes Step-Audio 2 a strong open reference for expressive, tool-using speech assistants.

Key points

  • Speech translation: On Chinese↔English tasks, Step-Audio 2 averages 39.26 BLEU on CoVoST 2 (speech-to-text), compared with 35.40 for Qwen2.5-Omni and 29.61 for GPT-4o Audio. On CVSS (speech-to-speech) it averages 30.87, compared with 23.68 for GPT-4o Audio.
  • Spoken dialogue (URO-Bench):
    • Chinese: it leads all compared models, scoring 83.32 in the basic track and 68.25 in the pro track.
    • English: GPT-4o Audio edges it out (84.54 vs 83.90 on the basic track).
  • New benchmark, StepEval-Audio-Toolcall:
    • Built from 200 synthesized Chinese multi-turn dialogues per tool, each 3–6 turns long, balanced with negative samples.
    • Judged automatically by Qwen3-32B.
    • Released on GitHub together with the scripts and evaluation code.
  • Tool calling: With speech input, Step-Audio 2 matches text-input Qwen3-32B on tool calling overall. It is clearly better at triggering audio search (precision 86.8 vs 67.5).
  • Model size: Step-Audio 2 has fewer parameters than its 130B predecessor Step-Audio. Code and further details are available at github.com/stepfun-ai/Step-Audio2.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 480 works

Cite

Cite as Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, Mingrui Chen, Peng Liu, Wang You, Xiangyu Tony Zhang, Xingyuan Li, Xuerui Yang, Yayue Deng, Yechang Huang, Yuxin Li, Yuxin Zhang, Zhao You, Brian Li, Changyi Wan, Hanpeng Hu, Jiangjie Zhen, Siyu Chen, Song Yuan, Xuelin Zhang, Yimin Jiang, Yu Zhou, Yuxiang Yang, Bingxin Li, Buyun Ma, Changhe Song, Dongqing Pang, Guoqiang Hu, Haiyang Sun, Kang An, Na Wang, Shuli Gao, Wei Ji, Wen Li, Wen Sun, Xuan Wen, Yong Ren, Yuankai Ma, Yufan Lu, Bin Wang, Bo Li, Changxin Miao, Che Liu, Chen Xu, Dapeng Shi, Dingyuan Hu, Donghang Wu, Enle Liu, Guanzhe Huang, Gulin Yan, Han Zhang, Hao Nie, Haonan Jia, Hongyu Zhou, Jianjian Sun, Jiaoren Wu, Jie Wu, Jie Yang, Jin Yang, Junzhe Lin, Kaixiang Li, Lei Yang, Liying Shi, Li Zhou, Longlong Gu, Ming Li, Mingliang Li, Mingxiao Li, Nan Wu, Qi Han, Qinyuan Tan, Shaoliang Pang, Shengjie Fan, Siqi Liu, Tiancheng Cao, Wanying Lu, Wenqing He, Wuxun Xie, Xu Zhao, Xueqi Li, Yanbo Yu, Yang Yang, Yi Liu, Yifan Lu, Yilei Wang, Yuanhao Ding, Yuanwei Liang, Yuanwei Lu, Yuchu Luo, Yuhe Yin, Yumeng Zhan, Yuxiang Zhang, Zidong Yang, Zixin Zhang, Binxing Jiao, Daxin Jiang, Heung-Yeung Shum, Jiansheng Chen, Jing Li, Xiangyu Zhang and Yibo Zhu. 2025. Step-Audio 2 Technical Report. arXiv:2507.16632.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2507.16632 · arXiv 2507.16632

@misc{wu2025step,
  title = {{Step-Audio 2 Technical Report}},
  author = {Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, Mingrui Chen, Peng Liu, Wang You, Xiangyu Tony Zhang, Xingyuan Li, Xuerui Yang, Yayue Deng, Yechang Huang, Yuxin Li, Yuxin Zhang, Zhao You, Brian Li, Changyi Wan, Hanpeng Hu, Jiangjie Zhen, Siyu Chen, Song Yuan, Xuelin Zhang, Yimin Jiang, Yu Zhou, Yuxiang Yang, Bingxin Li, Buyun Ma, Changhe Song, Dongqing Pang, Guoqiang Hu, Haiyang Sun, Kang An, Na Wang, Shuli Gao, Wei Ji, Wen Li, Wen Sun, Xuan Wen, Yong Ren, Yuankai Ma, Yufan Lu, Bin Wang, Bo Li, Changxin Miao, Che Liu, Chen Xu, Dapeng Shi, Dingyuan Hu, Donghang Wu, Enle Liu, Guanzhe Huang, Gulin Yan, Han Zhang, Hao Nie, Haonan Jia, Hongyu Zhou, Jianjian Sun, Jiaoren Wu, Jie Wu, Jie Yang, Jin Yang, Junzhe Lin, Kaixiang Li, Lei Yang, Liying Shi, Li Zhou, Longlong Gu, Ming Li, Mingliang Li, Mingxiao Li, Nan Wu, Qi Han, Qinyuan Tan, Shaoliang Pang, Shengjie Fan, Siqi Liu, Tiancheng Cao, Wanying Lu, Wenqing He, Wuxun Xie, Xu Zhao, Xueqi Li, Yanbo Yu, Yang Yang, Yi Liu, Yifan Lu, Yilei Wang, Yuanhao Ding, Yuanwei Liang, Yuanwei Lu, Yuchu Luo, Yuhe Yin, Yumeng Zhan, Yuxiang Zhang, Zidong Yang, Zixin Zhang, Binxing Jiao, Daxin Jiang, Heung-Yeung Shum, Jiansheng Chen, Jing Li, Xiangyu Zhang and Yibo Zhu},
  year = {2025},
  doi = {10.48550/arxiv.2507.16632},
  eprint = {2507.16632},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2507.16632}
}

References

82 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Philip Anastassiou et al. “Seed-tts: A family of high-quality versatile speech generation models”. In: arXiv preprint arXiv:2406.02430 (2024).
  2. Rohan Anil et al. PaLM 2 Technical Report. 2023. arXiv: 2305.10403 [cs.CL]. URL : https://arxiv.org/abs/2305.10403. ↗
  3. Alexei Baevski et al. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. 2020. arXiv: 2006.11477 [cs.CL]. URL : https://arxiv.org/abs/2006.11477. ↗
  4. Jinze Bai et al. Qwen Technical Report. 2023. arXiv: 2309.16609 [cs.CL]. URL : https://arxiv.org/abs/2309.16609. ↗
  5. Ye Bai et al. “Seed-asr: Understanding diverse speech and contexts with llm-based speech recognition”. In: arXiv preprint arXiv:2407.04675 (2024). — [arXiv:2407.04675](https://arxiv.org/abs/2407.04675) find
  6. James Betker. Better speech synthesis through scaling. 2023. arXiv: 2305 . 07243 [cs.SD]. URL : https : //arxiv.org/abs/2305.07243. — [arXiv:2305.07243](https://arxiv.org/abs/2305.07243) find
  7. Zalán Borsos et al. “Audiolm: a language modeling approach to audio generation”. In: IEEE/ACM transactions on audio, speech, and language processing 31 (2023), pp. 2523–2533.
  8. Guoguo Chen et al. “GigaSpeech: An Evolving, Multi-Domain ASR Corpus with 10,000 Hours of Transcribed Audio”. In: Interspeech 2021. ISCA, Aug. 2021. DOI : 10.21437/interspeech.2021- 1965. URL : http://dx.doi.org/10.21437/Interspeech.2021-1965. — [doi:10.21437/interspeech.2021-](https://doi.org/10.21437/interspeech.2021-) ↗
  9. Qian Chen et al. “Minmo: A multimodal large language model for seamless voice interaction”. In: arXiv preprint arXiv:2501.06282 (2025). — [arXiv:2501.06282](https://arxiv.org/abs/2501.06282) find
  10. Sanyuan Chen et al. BEATs: Audio Pre-Training with Acoustic Tokenizers. 2022. arXiv: 2212.09058 [eess.AS]. URL : https://arxiv.org/abs/2212.09058. — [arXiv:2212.09058](https://arxiv.org/abs/2212.09058) ↗
  11. Sanyuan Chen et al. “WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing”. In: IEEE Journal of Selected Topics in Signal Processing 16.6 (Oct. 2022), pp. 1505–1518. ISSN : 1941-0484. DOI : 10.1109/jstsp.2022.3188113. URL : http://dx.doi.org/10.1109/JSTSP.2022.3188113. ↗
  12. Yunfei Chu et al. “Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models”. In: arXiv preprint arXiv:2311.07919 (2023).
  13. Yunfei Chu et al. “Qwen2-audio technical report”. In: arXiv preprint arXiv:2407.10759 (2024).
  14. Jade Copet et al. Simple and Controllable Music Generation. 2024. arXiv: 2306.05284 [cs.SD]. URL : https://arxiv.org/abs/2306.05284. — [arXiv:2306.05284](https://arxiv.org/abs/2306.05284) ↗
  15. Alexandre Défossez et al. “High fidelity neural audio compression”. In: arXiv preprint arXiv:2210.13438 (2022).
  16. Alexandre Défossez et al. “Moshi: a speech-text foundation model for real-time dialogue”. In: arXiv preprint arXiv:2410.00037 (2024).
  17. Soham Deshmukh et al. “Pengi: An audio language model for audio tasks”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 18090–18108. find
  18. Ding Ding et al. “Kimi-audio technical report”. In: arXiv preprint arXiv:2504.18425 (2025).
  19. Zhihao Du et al. “Cosyvoice 2: Scalable streaming speech synthesis with large language models”. In: arXiv preprint arXiv:2412.10117 (2024). find
  20. Zhihao Du et al. CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens. 2024. arXiv: 2407.05407 [cs.SD]. URL : https://arxiv.org/abs/2407.05407. ↗
  21. Qingkai Fang et al. “Llama-omni: Seamless speech interaction with large language models”. In: arXiv preprint arXiv:2409.06666 (2024). find
  22. Heting Gao et al. LUCY: Linguistic Understanding and Control Yielding Early Stage of Her. 2025. arXiv: 2501.16327 [cs.CL]. URL : https://arxiv.org/abs/2501.16327. — [arXiv:2501.16327](https://arxiv.org/abs/2501.16327) ↗
  23. Jort F. Gemmeke et al. “Audio Set: An ontology and human-labeled dataset for audio events”. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2017, pp. 776–780. DOI : 10.1109/ICASSP.2017.7952261. — [doi:10.1109/icassp.2017.7952261](https://doi.org/10.1109/icassp.2017.7952261) find
  24. Sreyan Ghosh et al. Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities. 2025. arXiv: 2503.03983 [cs.SD]. URL : https://arxiv.org/abs/2503.03983. — [arXiv:2503.03983](https://arxiv.org/abs/2503.03983) ↗
  25. Arushi Goel et al. Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models. 2025. arXiv: 2507.08128 [cs.SD]. URL : https://arxiv.org/abs/2507.08128. — [arXiv:2507.08128](https://arxiv.org/abs/2507.08128) ↗
  26. Yuan Gong, Jin Yu, and James Glass. “Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition”. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2022, pp. 151–155. DOI : 10.1109/ICASSP43922.2022.9746828. — [doi:10.1109/icassp43922.2022.9746828](https://doi.org/10.1109/icassp43922.2022.9746828) find
  27. Yuan Gong et al. “Joint audio and speech understanding”. In: 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE. 2023, pp. 1–8. find
  28. Yuan Gong et al. “Listen, think, and understand”. In: arXiv preprint arXiv:2305.10790 (2023). — [arXiv:2305.10790](https://arxiv.org/abs/2305.10790) find
  29. Aaron Grattafiori et al. The Llama 3 Herd of Models. 2024. arXiv: 2407 . 21783 [cs.AI]. URL : https : //arxiv.org/abs/2407.21783.
  30. Wei-Ning Hsu et al. HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units. 2021. arXiv: 2106.07447 [cs.CL]. URL : https://arxiv.org/abs/2106.07447. ↗
  31. Ailin Huang et al. “Step-Audio-AQAA: a Fully End-to-End Expressive Large Audio Language Model”. In: arXiv preprint arXiv:2506.08967 (2025). — [arXiv:2506.08967](https://arxiv.org/abs/2506.08967) find
  32. Ailin Huang et al. “Step-audio: Unified understanding and generation in intelligent speech interaction”. In: arXiv preprint arXiv:2502.11946 (2025).
  33. Rongjie Huang et al. “Audiogpt: Understanding and generating speech, music, sound, and talking head”. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. 21. 2024, pp. 23802–23804. find
  34. Aaron Hurst et al. “Gpt-4o system card”. In: arXiv preprint arXiv:2410.21276 (2024).
  35. Il-Young Jeong and Jeongsoo Park. “CochlScene: Acquisition of acoustic scene data using crowdsourcing”. In: 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). 2022, pp. 17–21. DOI : 10.23919/APSIPAASC55919.2022.9979822. — [doi:10.23919/apsipaasc55919.2022.9979822](https://doi.org/10.23919/apsipaasc55919.2022.9979822) find
  36. Shengpeng Ji et al. “Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling”. In: arXiv preprint arXiv:2408.16532 (2024). — [arXiv:2408.16532](https://arxiv.org/abs/2408.16532) find
  37. Ye Jia et al. CVSS Corpus and Massively Multilingual Speech-to-Speech Translation. 2022. arXiv: 2201.03713 [cs.CL]. URL : https://arxiv.org/abs/2201.03713. ↗
  38. Ye Jia et al. “Direct speech-to-speech translation with a sequence-to-sequence model”. In: arXiv preprint arXiv:1904.06037 (2019). find
  39. Ye Jia et al. “Translatotron 2: High-quality direct speech-to-speech translation with voice preservation”. In: International conference on machine learning. PMLR. 2022, pp. 10120–10134. find
  40. Eugene Kharitonov et al. Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision. 2023. arXiv: 2302.03540 [cs.SD]. URL : https://arxiv.org/abs/2302.03540. ↗
  41. Chris Dongjoo Kim et al. “Audiocaps: Generating captions for audios in the wild”. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019, pp. 119–132.
  42. Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis. 2020. arXiv: 2010.05646 [cs.SD]. URL : https://arxiv.org/abs/2010.05646. ↗
  43. Zhifeng Kong et al. Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities. 2024. arXiv: 2402.01831 [cs.SD]. URL : https://arxiv.org/abs/2402.01831. — [arXiv:2402.01831](https://arxiv.org/abs/2402.01831) ↗
  44. Chenyang Le et al. “Transvip: Speech to speech translation system with voice and isochrony preservation”. In: Advances in Neural Information Processing Systems 37 (2024), pp. 89682–89705. find
  45. Ann Lee et al. “Textless speech-to-speech translation on real data”. In: arXiv preprint arXiv:2112.08352 (2021). find
  46. Sang-gil Lee et al. BigVGAN: A Universal Neural Vocoder with Large-Scale Training. 2023. arXiv: 2206.04658 [cs.SD]. URL : https://arxiv.org/abs/2206.04658. — [arXiv:2206.04658](https://arxiv.org/abs/2206.04658) ↗
  47. Guan-Ting Lin, Cheng-Han Chiang, and Hung-yi Lee. “Advancing large language models to capture varied speaking styles and respond properly in spoken conversations”. In: arXiv preprint arXiv:2402.12786 (2024). find
  48. Guan-Ting Lin et al. “Paralinguistics-enhanced large language modeling of spoken dialogue”. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, pp. 10316–10320. find
  49. Tu Anh Nguyen et al. Spirit LM: Interleaved Spoken and Written Language Model. 2024. arXiv: 2402.05755 [cs.CL]. URL : https://arxiv.org/abs/2402.05755. ↗
  50. OpenAI. GPT-4 Technical Report. https://openai.com/research/gpt-4. Accessed: 2025-07-11. 2023. ↗
  51. OpenAI. Introducing ChatGPT. Accessed: 2025-07-11. 2022. URL : https://openai.com/blog/chatgpt. ↗
  52. Wei Ping et al. “Deep voice 3: 2000-speaker neural text-to-speech”. In: proc. ICLR. Vol. 79. 2018, pp. 1094–1099. find
  53. Alec Radford et al. “Robust speech recognition via large-scale weak supervision”. In: International conference on machine learning. PMLR. 2023, pp. 28492–28518.
  54. Rafael Rafailov et al. “Direct preference optimization: Your language model is secretly a reward model”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 53728–53741.
  55. Yi Ren et al. “Fastspeech 2: Fast and high-quality end-to-end text to speech”. In: arXiv preprint arXiv:2006.04558 (2020). find
  56. Andrew Rouditchenko et al. “Omni-R1: Do You Really Need Audio to Fine-Tune Your Audio LLM?” In: arXiv preprint arXiv:2505.09439 (2025). — [arXiv:2505.09439](https://arxiv.org/abs/2505.09439) find
  57. Paul K Rubenstein et al. “Audiopalm: A large language model that can speak and listen”. In: arXiv preprint arXiv:2306.12925 (2023).
  58. S Sakshi et al. “Mmau: A massive multi-task audio understanding and reasoning benchmark”. In: arXiv preprint arXiv:2410.19168 (2024). find
  59. Jonathan Shen et al. “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions”. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE. 2018, pp. 4779–4783. find
  60. Hubert Siuzdak, Florian Grötschla, and Luca A Lanzendörfer. “Snac: Multi-scale neural audio codec”. In: arXiv preprint arXiv:2410.14411 (2024). — [arXiv:2410.14411](https://arxiv.org/abs/2410.14411) find
  61. Changli Tang et al. “Salmonn: Towards generic hearing abilities for large language models”. In: arXiv preprint arXiv:2310.13289 (2023).
  62. Wolfgang Wahlster. Verbmobil: foundations of speech-to-speech translation. Springer Science & Business Media, 2013. find
  63. Changhan Wang, Anne Wu, and Juan Pino. CoVoST 2 and Massively Multilingual Speech-to-Text Translation. 2020. arXiv: 2007.10310 [cs.CL]. URL : https://arxiv.org/abs/2007.10310. ↗
  64. Chengyi Wang et al. “Neural codec language models are zero-shot text to speech synthesizers”. In: arXiv preprint arXiv:2301.02111 (2023).
  65. Xinsheng Wang et al. “Spark-tts: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens”. In: arXiv preprint arXiv:2503.01710 (2025). — [arXiv:2503.01710](https://arxiv.org/abs/2503.01710) find
  66. Xiong Wang et al. “Freeze-omni: A smart and low latency speech-to-speech dialogue model with frozen llm”. In: arXiv preprint arXiv:2411.00774 (2024). find
  67. Yuancheng Wang et al. “Maskgct: Zero-shot text-to-speech with masked generative codec transformer”. In: arXiv preprint arXiv:2409.00750 (2024). — [arXiv:2409.00750](https://arxiv.org/abs/2409.00750) find
  68. Yuxuan Wang et al. “Tacotron: Towards end-to-end speech synthesis”. In: arXiv preprint arXiv:1703.10135 (2017). find
  69. Jason Wei et al. “Finetuned language models are zero-shot learners”. In: arXiv preprint arXiv:2109.01652 (2021).
  70. Yonghui Wu et al. “Google’s neural machine translation system: Bridging the gap between human and machine translation”. In: arXiv preprint arXiv:1609.08144 (2016). find
  71. Zhifei Xie and Changqiao Wu. “Mini-omni: Language models can hear, talk while thinking in streaming”. In: arXiv preprint arXiv:2408.16725 (2024). find
  72. Zhifei Xie and Changqiao Wu. “Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities”. In: arXiv preprint arXiv:2410.11190 (2024). — [arXiv:2410.11190](https://arxiv.org/abs/2410.11190) find
  73. Detai Xin et al. “Bigcodec: Pushing the limits of low-bitrate neural speech codec”. In: arXiv preprint arXiv:2409.05377 (2024). — [arXiv:2409.05377](https://arxiv.org/abs/2409.05377) find
  74. Jin Xu et al. Qwen2.5-Omni Technical Report. 2025. arXiv: 2503.20215 [cs.CL]. URL : https://arxiv.org/abs/2503.20215. ↗
  75. Ruiqi Yan et al. URO-Bench: A Comprehensive Benchmark for End-to-End Spoken Dialogue Models. 2025. arXiv: 2502.17810 [cs.CL]. URL : https://arxiv.org/abs/2502.17810. — [arXiv:2502.17810](https://arxiv.org/abs/2502.17810) ↗
  76. Neil Zeghidour et al. “Soundstream: An end-to-end neural audio codec”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 30 (2021), pp. 495–507.
  77. Aohan Zeng et al. “Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot”. In: arXiv preprint arXiv:2412.02612 (2024).
  78. Binbin Zhang et al. WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognition. 2022. arXiv: 2110.03370 [cs.SD]. URL : https://arxiv.org/abs/2110.03370. — [arXiv:2110.03370](https://arxiv.org/abs/2110.03370) ↗
  79. Bowen Zhang et al. “Minimax-speech: Intrinsic zero-shot text-to-speech with a learnable speaker encoder”. In: arXiv preprint arXiv:2505.07916 (2025). — [arXiv:2505.07916](https://arxiv.org/abs/2505.07916) find
  80. Dong Zhang et al. “Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities”. In: arXiv preprint arXiv:2305.11000 (2023).
  81. Xiangyu Zhang et al. “Distinctive Feature Codec: Adaptive Segmentation for Efficient Speech Representation”. In: arXiv preprint arXiv:2505.18516 (2025). — [arXiv:2505.18516](https://arxiv.org/abs/2505.18516) find
  82. Xin Zhang et al. “Speechtokenizer: Unified speech tokenizer for speech large language models”. In: arXiv preprint arXiv:2308.16692 (2023).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2025-step-audio-2-technical/