Elamur Research⌕
☰
home / papers / 2026 / VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation

VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation

Paper · 2026 · Puyuan Peng, Zhisheng Zheng, Shang-Wen Li et al.

Summary

Autoregressive neural codec language model (NCLM) TTS systems usually concatenate text and speech tokens and leave alignment to ordinary positional encodings. As a result, they cannot easily be told how long the output should be, and they degrade when asked to produce speech longer than their training clips. VoiceStar addresses both problems. It is an encoder-decoder NCLM: IPA phonemes go into the encoder, and the decoder predicts Encodec speech tokens. Its central idea, Progress-Monitoring RoPE (PM-RoPE), places text and speech positions on a shared scale tied to the target length. This gives the model an early rough alignment, a way to specify duration, and the ability to run past the training length. Continuation-Prompt Mixed (CPM) training sometimes uses a different utterance from the same speaker as the prompt, which reduces the gap between training and inference. It also makes prompt speed perturbation possible. On short-form benchmarks VoiceStar is competitive with the state of the art. On 30–50 s generation it clearly beats F5-TTS and MaskGCT, which suggests length extrapolation for zero-shot voice cloning is achievable without long-form training data.

Key points

  • The authors describe VoiceStar as the first zero-shot TTS model offering both duration control and extrapolation beyond training length. All compared models were trained on clips of at most 30 s.
  • On the 40–50 s extrapolation range, VoiceStar's WER is 11.91, compared with 52.44 for F5-TTS and 82.29 for MaskGCT. On the 30–40 s range, it is 7.27 versus 34.15 and 13.81.
  • On short-form benchmarks, VoiceStar reaches a WER of 2.64 on LibriSpeech-PC and 2.15 on Seed-TTS (en), the latter matching ground truth. It also has the best speaker-similarity CMOS on LibriSpeech-PC (0.60).
  • Additional techniques include speed perturbation of the prompt during training and prompt repetition at inference. Ablations show that mixing continuation and same-speaker prompts works better than either prompting style alone.
  • Limitations: speaker similarity on long-form output lags slightly behind the best models, and the 840M model generates slower than real time (RTF > 1). Code and model are available at github.com/jasonppy/VoiceStar.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of Texas at AustinAustin · United States109 works · 56,942 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 worksMedicine · 241 worksPhysiology · 29 works

Cite

Cite as Puyuan Peng, Zhisheng Zheng, Shang-Wen Li, Abdelrahman Mohamed and David Harwath. 2026. VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.570

@inproceedings{peng2026voicestar,
  title = {{VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation}},
  author = {Puyuan Peng, Zhisheng Zheng, Shang-Wen Li, Abdelrahman Mohamed and David Harwath},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.570},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.570}
}

References

82 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li, Hui Li, and 27 others. 2024. Seed-tts: A family of high-quality ve…
  2. Eric Battenberg, R. J. Skerry-Ryan, Daisy Stanton, Soroosh Mariooryad, Matt Shannon, Julian Salazar, and David Kao. 2024. Very attentive tacotron: Robust and unbounded length generalization in autoregressive transformer-based text-to-speech. ArXiv, abs/2410.22179. find
  3. Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matthew Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. 2022. Audiolm: A language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and L…
  4. Zalán Borsos, Matthew Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. 2023. Soundstorm: Efficient parallel audio generation. ArXiv, abs/2305.09636. — [arXiv:2305.09636](https://arxiv.org/abs/2305.09636) find
  5. Chen Chen, Yuchen Hu, Wen Wu, Helin Wang, Chng Eng Siong, and Chao Zhang. 2024a. Enhancing zero-shot text-to-speech synthesis with human feedback. ArXiv, abs/2406.00654. — [arXiv:2406.00654](https://arxiv.org/abs/2406.00654) find
  6. Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, and Furu Wei. 2024b. Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers. Preprint, arXiv:2406.05370. — [arXiv:2406.05370](https://arxiv.org/abs/2406.05370) find
  7. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Micheal Zeng, and Furu Wei. 2021. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEE… find
  8. Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. 2024c. F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching. ArXiv, abs/2410.06885.
  9. Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Defossez. 2023. Simple and controllable music generation. ArXiv, abs/2306.05284. — [arXiv:2306.05284](https://arxiv.org/abs/2306.05284) find
  10. Alexandre Defossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. High fidelity neural audio compression. ArXiv, abs/2210.13438.
  11. Alexandre D’efossez, Laurent Mazar’e, Manu Orsini, Am’elie Royer, Patrick P’erez, Herv’e J’egou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: a speech-text foundation model for real-time dialogue. ArXiv, abs/2410.00037.
  12. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Llm.int8(): 8-bit matrix multiplication for transformers at scale. Preprint, arXiv:2208.07339. find
  13. Anuj Diwan, Zhisheng Zheng, David Harwath, and Eunsol Choi. 2025. Scaling rich style-prompted text-to-speech datasets. find
  14. Chenpeng Du, Yiwei Guo, Hankun Wang, Yifan Yang, Zhikang Niu, Shuai Wang, Hui Zhang, Xie Chen, and Kai Yu. 2024a. Vall-t: Decoder-only generative transducer for robust and decoding-controllable text-to-speech. find
  15. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao, and Zhijie Yan. 2024b. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. ArXiv, abs/2407.05407. find
  16. Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, Fan Yu, Huadai Liu, Zhengyan Sheng, Yue Gu, Chong Deng, Wen Wang, Shiliang Zhang, Zhijie Yan, and Jing-Ru Zhou. 2024c. Cosyvoice 2: Scalable streaming speech synthesis with large langu… find
  17. Jonathan Duddington, Reece Dunn, and the eSpeak NG Community. eSpeak NG: Speech synthesiser. https://github.com/espeak-ng/ espeak-ng. Accessed: March 15, 2025. ↗
  18. Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, Yanqing Liu, Sheng Zhao, and Naoyuki Kanda. 2024. E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts. 2024 IEEE Spoken Language Technology Workshop (SLT… find
  19. Qingkai Fang, Shoutao Guo, Yan Zhou, Zhengrui Ma, Shaolei Zhang, and Yang Feng. 2024. Llama-omni: Seamless speech interaction with large language models. ArXiv, abs/2409.06666. find
  20. Hao-Han Guo, Kun Liu, Fei-Yu Shen, Yi-Chen Wu, Fenglong Xie, Kun Xie, and Kai-Tuo Xu. 2024a. Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications. ArXiv, abs/2409.03283. — [arXiv:2409.03283](https://arxiv.org/abs/2409.03283) find
  21. Haohan Guo, Fenglong Xie, Dongchao Yang, Xixin Wu, and Helen M. Meng. 2024b. Speaking from coarse to fine: Improving neural codec language model via multi-scale speech coding and generation. ArXiv, abs/2409.11630. — [arXiv:2409.11630](https://arxiv.org/abs/2409.11630) find
  22. Zhifang Guo, Yichong Leng, Yihan Wu, Sheng Zhao, and Xuejiao Tan. 2022. Prompttts: Controllable text-to-speech with text descriptions. find
  23. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. find
  24. Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanmin Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, and Furu Wei. 2024. Vall-e r: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment. ArXiv, abs/2406.07855. — [arXiv:2406.07855](https://arxiv.org/abs/2406.07855) find
  25. Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, and Zhizheng Wu. 2024. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation. 2024 IEEE Spoken L… find
  26. Yuchen Hu, Chen Chen, Siyin Wang, Chng Eng Siong, and Chao Zhang. 2024. Robust zero-shot text-to-speech synthesis with reverse inference optimization. ArXiv, abs/2407.02243. — [arXiv:2407.02243](https://arxiv.org/abs/2407.02243) find
  27. Shehzeen Samarah Hussain, Paarth Neekhara, Xuesong Yang, Edresson Casanova, Subhankar Ghosh, Mikyas T. Desta, Roy Fejgin, Rafael Valle, and Jason Li. 2025. Koel-tts: Enhancing llm based speech generation with preference alignment and classifier free guidance. find
  28. Shengpeng Ji, Jia li Zuo, Minghui Fang, Ziyue Jiang, Feiyang Chen, Xinyu Duan, Baoxing Huai, and Zhou Zhao. 2023. Textrolspeech: A text style control speech corpus with codec language text-to-speech models. ArXiv, abs/2308.14430. — [arXiv:2308.14430](https://arxiv.org/abs/2308.14430) find
  29. Ziyue Jiang, Jinglin Liu, Yi Ren, Jinzheng He, Zhenhui Ye, Shengpeng Ji, Qian Yang, Chen Zhang, Pengfei Wei, Chunfeng Wang, Xiang Yin, Zejun MA, and Zhou Zhao. 2024. Mega-TTS 2: Boosting prompting mechanisms for zero-shot speech synthesis. In The Twelfth International Conference on Learning Represen… find
  30. Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Yifan Yang, Liyong Guo, Long Lin, and Daniel Povey. 2023. Libriheavy: A 50,000 hours asr corpus with punctuation casing and context. find
  31. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10991–10995. find
  32. Eugene Kharitonov, Damien Vincent, Zalán Borsos, Raphaël Marinier, Sertan Girgin, Olivier Pietquin, Matthew Sharifi, Marco Tagliasacchi, and Neil Zeghidour. 2023. Speak, read and prompt: High-fidelity text-to-speech with minimal supervision. Transactions of the Association for Computational Linguist…
  33. Sungwon Kim, Kevin J. Shih, Rohan Badlani, João Felipe Santos, Evelina Bakhturina, Mikyas T. Desta, Rafael Valle, Sungroh Yoon, and Bryan Catanzaro. 2023. P-flow: A fast and data-efficient zero-shot tts through speech prompting. In Neural Information Processing Systems. find
  34. Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Defossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. 2022. Audiogen: Textually guided audio generation. ArXiv, abs/2209.15352. find
  35. Mateusz Lajszczak, Guillermo Cámbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, Álvaro Martín-Cortinas, Ammar Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszy’nska, Haohan Guo, Bartosz Putrycz, Soledad López Gambino, Kayeon Yoo, Elena Sokolova, and Thomas Drugman… — [arXiv:2402.08093](https://arxiv.org/abs/2402.08093) find
  36. Kushal Lakhotia, Evgeny Kharitonov, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu Nguyen, Jade Copet, Alexei Baevski, Adel Ben Mohamed, and Emmanuel Dupoux. 2021. On generative spoken language modeling from raw audio. Transactions of the Association for Computational Linguistics, 9:1336–1…
  37. Matt Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. ArXiv, abs/2306.15687. — [arXiv:2306.15687](https://arxiv.org/abs/2306.15687) find
  38. Keon Lee, Dong Won Kim, Jaehyeon Kim, Seungjun Chung, and Jaewoong Cho. 2024. Ditto-tts: Diffusion transformers for scalable text-to-speech without domain-specific factors. find
  39. Yichong Leng, Zhifang Guo, Kai Shen, Xu Tan, Zeqian Ju, Yanqing Liu, Yufei Liu, Dongchao Yang, Leying Zhang, Kaitao Song, Lei He, Xiang-Yang Li, Sheng Zhao, Tao Qin, and Jiang Bian. 2023. Prompttts 2: Describing and generating voices with text prompt. ArXiv, abs/2309.02285. — [arXiv:2309.02285](https://arxiv.org/abs/2309.02285) find
  40. Bohan Li, Hankun Wang, Situo Zhang, Yiwei Guo, and Kai Yu. 2025. Fast and high-quality autoregressive speech synthesis via speculative decoding. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  41. Yingahao Aaron Li, Rithesh Kumar, and Zeyu Jin. 2024. Dmospeech: Direct metric optimization via distilled diffusion model in zero-shot speech synthesis. find
  42. Guanghou Liu, Yongmao Zhang, Yinjiao Lei, Yunlin Chen, Rui Wang, Zhifei Li, and Linfu Xie. 2023. Promptstyle: Controllable style transfer for text-to-speech with natural language descriptions. ArXiv, abs/2305.19522. — [arXiv:2305.19522](https://arxiv.org/abs/2305.19522) find
  43. Philipos C Loizou. 2011. Speech quality assessment. In Multimedia analysis, processing and communications, pages 623–654. Springer. find
  44. Daniel Lyth and Simon King. 2024. Natural language guidance of high-fidelity text-to-speech with synthetic annotations. ArXiv, abs/2402.01912. — [arXiv:2402.01912](https://arxiv.org/abs/2402.01912) find
  45. Ziyang Ma, Ya-Zhen Song, Chenpeng Du, Jian Cong, Zhuo Chen, Yuping Wang, Yuxuan Wang, and Xie Chen. 2024. Language model can listen while speaking. ArXiv, abs/2408.02622. — [arXiv:2408.02622](https://arxiv.org/abs/2408.02622) find
  46. Soumi Maiti, Yifan Peng, Shukjae Choi, Jee weon Jung, Xuankai Chang, and Shinji Watanabe. 2023. Voxtlm: Unified decoder-only models for consolidating speech recognition, synthesis and speech, text continuation tasks. find
  47. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 13326–13330. find
  48. Paarth Neekhara, Shehzeen Samarah Hussain, Subhankar Ghosh, Jason Li, Rafael Valle, Rohan Badlani, and Boris Ginsburg. 2024. Improving robustness of llm-based speech synthesis by learning monotonic alignment. ArXiv, abs/2406.17957. — [arXiv:2406.17957](https://arxiv.org/abs/2406.17957) find
  49. Tan Dat Nguyen, Ji-Hoon Kim, Jeongsoo Choi, Shukjae Choi, Jinseok Park, Younglo Lee, and Joon Son Chung. 2025. Accelerating codec-based speech synthesis with multi-token prediction and speculative decoding. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing… find
  50. Yuto Nishimura, Takumi Hirose, Masanari Ohi, Hideki Nakayama, and Nakamasa Inoue. 2024. Hall-e: Hierarchical neural codec language model for minute-long zero-shot text-to-speech synthesis. ArXiv, abs/2410.04380. — [arXiv:2410.04380](https://arxiv.org/abs/2410.04380) find
  51. Puyuan Peng, Po-Yao Huang, Shang-Wen Li, Abdelrahman Mohamed, and David Harwath. 2024. Voicecraft: Zero-shot speech editing and text-to-speech in the wild. In Annual Meeting of the Association for Computational Linguistics. find
  52. Ofir Press, Noah A. Smith, and Mike Lewis. 2022. Train short, test long: Attention with linear biases enables input length extrapolation. Preprint, arXiv:2108.12409. find
  53. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. ArXiv, abs/2212.04356.
  54. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2023. Exploring the limits of transfer learning with a unified text-to-text transformer. Preprint, arXiv:1910.10683.
  55. Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. Utmos: Utokyo-sarulab system for voicemos challenge 2022. Preprint, arXiv:2204.02152. find
  56. Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. 2023. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. ArXiv, abs/2304.09116. — [arXiv:2304.09116](https://arxiv.org/abs/2304.09116) find
  57. Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, and Xie Chen. 2024. Ella-v: Stable neural codec language modeling with alignment-guided sequence reordering. find
  58. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. 2023. Roformer: Enhanced transformer with rotary position embedding. Preprint, arXiv:2104.09864.
  59. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. Attention is all you need. Preprint, arXiv:1706.03762.
  60. Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Neural Information Processing Systems.
  61. Apoorv Vyas, Bowen Shi, Matt Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, W.K.F. Ngan, Jeff Wang, Ivan Cruz, Bapi Akula, Akinniyi Tunde Akinyemi, Brian Ellis, Rashel Moritz, Yael Yungster, Alice Rakotoarison, Liang Tan, and 5 others. 2023. Audiobox: Unifie… — [arXiv:2312.15821](https://arxiv.org/abs/2312.15821) find
  62. Chengyi Wang, Sanyuan Chen, Yu Wu, Zi-Hua Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2023a. Neural codec language models are zero-shot text to speech synthesizers. ArXiv, abs/2301.02111.
  63. Hankun Wang, Chenpeng Du, Yiwei Guo, Shuai Wang, Xie Chen, and Kai Yu. 2024a. Attention-constrained inference for robust decoder-only text-to-speech. 2024 IEEE Spoken Language Technology Workshop (SLT), pages 630–637. find
  64. Helin Wang, Meng Yu, Jiarui Hai, Chen Chen, Yuchen Hu, Rilin Chen, Najim Dehak, and Dong Yu. 2024b. Ssr-speech: Towards stable, safe and robust zero-shot text-based speech editing and synthesis. ArXiv, abs/2409.07556. — [arXiv:2409.07556](https://arxiv.org/abs/2409.07556) find
  65. Jiaming Wang, Zhihao Du, Qian Chen, Yunfei Chu, Zhifu Gao, Zerui Li, Kai Hu, Xiaohuan Zhou, Jin Xu, Ziyang Ma, Wen Wang, Siqi Zheng, Chang Zhou, Zhijie Yan, and Shiliang Zhang. 2023b. Lauragpt: Listen, attend, understand, and regenerate audio with gpt. ArXiv, abs/2310.04673. — [arXiv:2310.04673](https://arxiv.org/abs/2310.04673) find
  66. Tianrui Wang, Long Zhou, Zi-Hua Zhang, Yu Wu, Shujie Liu, Yashesh Gaur, Zhuo Chen, Jinyu Li, and Furu Wei. 2023c. Viola: Unified codec language models for speech recognition, synthesis, and translation. ArXiv, abs/2305.16107. — [arXiv:2305.16107](https://arxiv.org/abs/2305.16107) find
  67. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, Weizhen Bian, Zhen Ye, Sitong Cheng, Ruibin Yuan, Zhixian Zhao, Xinfa Zhu, Jiahao Pan, Liumeng Xue, Pengcheng Zhu, and 6 others. 2025a. Spark-tts: An efficient llm-based tex… find
  68. Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. 2024c. Maskgct: Zero-shot text-to-speech with masked generative codec transformer. ArXiv, abs/2409.00750. — [arXiv:2409.00750](https://arxiv.org/abs/2409.00750) find
  69. Yuancheng Wang, Jiachen Zheng, Junan Zhang, Xueyao Zhang, Huan Liao, and Zhizheng Wu. 2025b. Metis: A foundation speech generation model with masked generative pre-training. find
  70. Yihan Wu, Soumi Maiti, Yifan Peng, Wangyou Zhang, Chenda Li, Yuyue Wang, Xihua Wang, Shinji Watanabe, and Ruihua Song. 2024. Speechcomposer: Unifying multiple speech tasks with prompt composition. ArXiv, abs/2401.18045. — [arXiv:2401.18045](https://arxiv.org/abs/2401.18045) find
  71. Detai Xin, Xu Tan, Kai Shen, Zeqian Ju, Dongchao Yang, Yuancheng Wang, Shinnosuke Takamichi, Hiroshi Saruwatari, Shujie Liu, Jinyu Li, and Sheng Zhao. 2024. Rall-e: Robust codec language modeling with chain-of-thought prompting for text-to-speech synthesis. ArXiv, abs/2404.03204. — [arXiv:2404.03204](https://arxiv.org/abs/2404.03204) find
  72. Wang Xu, Shuo Wang, Weilin Zhao, Xu Han, Yukun Yan, Yudi Zhang, Zhe Tao, Zhiyuan Liu, and Wanxiang Che. 2024. Enabling real-time conversations with minimal training costs. ArXiv, abs/2409.11727. find
  73. Dongchao Yang, Songxiang Liu, Rongjie Huang, Guangzhi Lei, Chao Weng, Helen M. Meng, and Dong Yu. 2023. Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt. ArXiv, abs/2301.13662. — [arXiv:2301.13662](https://arxiv.org/abs/2301.13662) find
  74. Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. 2024. Zipformer: A faster and better encoder for automatic speech recognition. In ICLR. find
  75. Zhen Ye, Xinfa Zhu, Chi min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi Dai, Hongzhan Lin, Jianyi Chen, Xingjian Du, Liumeng Xue, Yunlin Chen, Zhifei Li, Lei Xie, Qiuqiang Kong, Yi-Ting Guo, and Wei Xue. 2025. Llasa: Scaling train-time and inference-time compute for… find
  76. Wenyi Yu, Siyin Wang, Xiaoyu Yang, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Guangzhi Sun, Lu Lu, Yuxuan Wang, and Chao Zhang. 2024. Salmonn-omni: A codec-free llm for full-duplex speech understanding and generation. ArXiv, abs/2411.18138. — [arXiv:2411.18138](https://arxiv.org/abs/2411.18138) find
  77. Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507.
  78. Hao Zhang, Weiwei Li, Rilin Chen, Vinay Kothapally, Meng Yu, and Dong Yu. 2025. Llm-enhanced dialogue management for full-duplex spoken dialogue systems. find
  79. Xin Zhang, Xiang Lyu, Zhihao Du, Qian Chen, Dong Zhang, Hangrui Hu, Chaohong Tan, Tianyu Zhao, Yuxuan Wang, Bin Zhang, Heng Lu, Yaqian Zhou, and Xipeng Qiu. 2024. Intrinsicvoice: Empowering llms with intrinsic real-time voice interaction abilities. ArXiv, abs/2410.08035. — [arXiv:2410.08035](https://arxiv.org/abs/2410.08035) find
  80. Zi-Hua Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2023. Speak foreign languages with your own voice: Cross-lingual neural codec language modeling. ArXiv, abs/2303.03926. — [arXiv:2303.03926](https://arxiv.org/abs/2303.03926) find
  81. Zhisheng Zheng, Puyuan Peng, Anuj Diwan, Cong Phuoc Huynh, Xiaohang Sun, Zhu Liu, Vimal Bhat, and David Harwath. 2025. Voicecraft-x: Unifying multilingual, voice-cloning speech synthesis and speech editing. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pa… find
  82. Yongxin Zhu, Dan Su, Liqiang He, Linli Xu, and Dong Yu. 2024. Generative pre-trained speech language model with efficient hierarchical transformer. ArXiv, abs/2406.00976. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/peng-2026-voicestar-robust-zero-shot-autoregressive/