Elamur Research⌕
☰
home / papers / 2026 / WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing Ecosystem

WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing Ecosystem

Paper · 2026 · Chengyou Wang, Mingchen Shao, Jingbin Hu et al.

Summary

Wu Chinese has roughly 100 million speakers and is linguistically complex, with voiced initials and intricate tone sandhi. Yet speech technology for it has been held back by scarce data, no shared benchmarks, and weak models. Before this work, the only open dataset, MagicData-Shanghai, held about 4 hours of Shanghainese ASR data. The authors address this with three releases. WenetSpeech-Wu is an open corpus of about 8,000 hours covering eight Wu sub-dialects. It was built with an automatic pipeline that collects, filters, transcribes, and labels in-the-wild audio, adding translations, speaker attributes, emotion, and prosodic features. WenetSpeech-Wu-Bench is a manually labeled benchmark spanning ASR, Wu-to-Mandarin AST, gender and age prediction, emotion recognition, TTS, and instruct TTS. The third release comprises models trained on the corpus: ASR and TTS models, plus models for instruction-following TTS and for speech understanding within one unified framework. The authors report that these models match or outperform existing open and commercial systems on most measures. Together, the releases give Wu a full data, evaluation, and modeling ecosystem, which supports reproducible research on this under-served dialect.

Key points

  • The corpus contains about 8k hours across 11 domains and 8 Wu sub-dialects. The prior open resource, MagicData-Shanghai, offered only 4.19 hours.
  • The pipeline begins with metadata filtering, WebRTC VAD segmentation, and DNSMOS/SNR quality filtering. Transcription uses Tele-CTC-FT and Step-Audio2-FT, both fine-tuned on 880 hours of manually transcribed Wu speech. Qwen3-8b handles cross-validation and refinement.
  • TTS training starts from CosyVoice2 and proceeds in three stages: CPT on TTS-Mid for 10 epochs, SFT on TTS-High for 3 epochs, and single-speaker fine-tuning on 10 hours of internal data. The resulting CosyVoice2-Wu-SS approaches or beats Qwen3-TTS, DiaMoE-TTS, and CosyVoice2 on most metrics.
  • Instruction tuning on the Inst Pro and Inst Emo data improves all controllability metrics. For example, EMOS rises from 3.66 to 3.83 and PMOS from 2.13 to 3.68.
  • The unified understanding model outperforms Qwen3-Omni on age and emotion prediction but is slightly weaker on gender. All resources are released for non-commercial research via the ASLP-lab GitHub repository.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Xi’an Jiaotong-Liverpool UniversitySuzhou · China10 works · 319 citations🎓Northwestern Polytechnical UniversityXi'an · China13 works · 67 citations

Also on the byline: San Francisco Bay University · WNET.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 worksSpeech & spoken language · 480 works

Cite

Cite as Chengyou Wang, Mingchen Shao, Jingbin Hu, Zeyu Zhu, Hongfei Xue, Bingshen Mu, Xin Xu, Xingyi Duan, Binbin Zhang, Zhu Pengcheng, Chuang Ding, Xiaojun Zhang, Hui Bu and Lei Xie. 2026. WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing Ecosystem. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1395

@inproceedings{wang2026wenetspeech,
  title = {{WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing Ecosystem}},
  author = {Chengyou Wang, Mingchen Shao, Jingbin Hu, Zeyu Zhu, Hongfei Xue, Bingshen Mu, Xin Xu, Xingyi Duan, Binbin Zhang, Zhu Pengcheng, Chuang Ding, Xiaojun Zhang, Hui Bu and Lei Xie},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1395},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1395}
}

References

37 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Keyu An, Qian Chen, Chong Deng, Zhihao Du, Changfeng Gao, Zhifu Gao, Yue Gu, Ting He, Hangrui Hu, Kai Hu, Shengpeng Ji, Yabin Li, Zerui Li, Heng Lu, Haoneng Luo, Xiang Lv, Bin Ma, Ziyang Ma, Chongjia Ni, and 14 others. 2024. Funaudiollm: Voice understanding and generation foundation models for natur… — [arXiv:2407.04051](https://arxiv.org/abs/2407.04051) find
  2. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proc. LREC, pages 4218–4222.
  3. Hervé Bredin. 2023. pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe. In Proc. INTERSPEECH, pages 1983–1987. find
  4. Cedric Chan and Jianjing Kuang. 2025. Toward Objective and Interpretable Prosody Evaluation in Text-to-Speech: A Linguistically Motivated Approach. arXiv, 2511.02104. find
  5. Hongjie Chen, Zehan Li, Guangmin Xia, Boqing Liu, Yan Yang, Jian Kang, and Jie Li. 2024. Tele-SpeechPT: Large-Scale Chinese Multi-dialect and Multi-accent Speech Pre-training, pages 183–190. Springer. find
  6. Ziqi Chen, Gongyu Chen, Yihua Wang, Chaofan Ding, Zihao Chen, and Wei-Qiang Zhang. 2025. Diamoe-tts: A unified ipa-based dialect TTS framework with mixture-of-experts and parameter-efficient zero-shot adaptation. CoRR, abs/2509.22727. — [arXiv:2509.22727](https://arxiv.org/abs/2509.22727) find
  7. Deok-Hyeon Cho, Hyung-Seok Oh, Seung-Bin Kim, and Seong-Whan Lee. 2025. Emosphere++: Emotion-controllable zero-shot text-to-speech via emotion-adaptive spherical vector. IEEE Trans. Affect. Comput., 16(3):2365–2380. find
  8. Yuhang Dai, Ziyu Zhang, Shuai Wang, Longhao Li, Zhao Guo, Tianlun Zuo, Shuiyuan Wang, Hongfei Xue, Chengyou Wang, Qing Wang, Xin Xu, Hui Bu, Jie Li, Jian Kang, Binbin Zhang, and Lei Xie. 2025. Wenetspeech-chuan: A large-scale sichuanese corpus with rich annotation for dialectal speech processing. Co… — [arXiv:2509.18004](https://arxiv.org/abs/2509.18004) find
  9. J.G. Fiscus. 1997. A post-processing system to yield reduced word error rates: Recognizer output voting error reduction (rover). In Proc. ASRU. find
  10. Changfeng Gao, Zhihao Du, and Shiliang Zhang. 2025. Differentiable reward optimization for LLM based TTS system. In Proc. Interspeech. find
  11. Zhifu Gao, Shiliang Zhang, Ian McLoughlin, and Zhijie Yan. 2022. Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition. In Proc. Interspeech, pages 2063–2067. find
  12. Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, and Zhizheng Wu. 2024. Emilia: An Extensive, Multilingual, and Diverse Speech Dataset For Large-Scale Speech Generation. In Proc. SLT, page… find
  13. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. CoRR, abs/2106.09685.
  14. Chun-Yi Kuan, Chen-An Li, Tsu-Yuan Hsu, Tse-Yang Lin, Ho-Lam Chung, Kai-Wei Chang, Shuo-Yiin Chang, and Hung-Yi Lee. 2023. Towards general-purpose text-instruction-guided voice conversion. In Proc. ASRU, pages 1–8. find
  15. Longhao Li, Zhao Guo, Hongjie Chen, Yuhang Dai, Ziyu Zhang, Hongfei Xue, Tianlun Zuo, Chengyou Wang, Shuiyuan Wang, Jie Li, Jian Kang, Xin Xu, Hui Bu, Binbin Zhang, Ruibin Yuan, Ziya Zhou, Wei Xue, and Lei Xie. 2025. Wenetspeech-yue: A large-scale cantonese speech corpus with multidimensional annota… — [arXiv:2509.03959](https://arxiv.org/abs/2509.03959) find
  16. Daniel Lyth and Simon King. 2024. Natural language guidance of high-fidelity text-to-speech with synthetic annotations. CoRR, abs/2402.01912. — [arXiv:2402.01912](https://arxiv.org/abs/2402.01912) find
  17. Yangyang Meng, Jinpeng Li, Guodong Lin, Yu Pu, Guanbo Wang, Hu Du, Zhiming Shao, Yukai Huang, Ke Li, and Wei-Qiang Zhang. 2025. Dolphin: A large-scale automatic speech recognition model for eastern languages. CoRR, abs/2503.20212. — [arXiv:2503.20212](https://arxiv.org/abs/2503.20212) find
  18. Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, Fan Yu, Huadai Liu, Zhengyan Sheng, Yue Gu, Chong Deng, Wen Wang, Shiliang Zhang, Zhijie Yan, and Jingren Zhou. 2024. Cosyvoice 2: Scalable streaming speech synthesis with large langua… find
  19. Jing Peng, Yucheng Wang, Bohan Li, Yiwei Guo, Hankun Wang, YanGui Fang, Yu Xi, Haoyu Li, Xu Li, Ke Zhang, Shuai Wang, and Kai Yu. 2025. A Survey on Speech Large Language Models for Understanding. IEEE Journal of Selected Topics in Signal Processing, page 1–32. find
  20. Tiantian Feng, Jihwan Lee, Anfeng Xu, Yoonjeong Lee, Thanathai Lertpetchpun, Xuan Shi, Helin Wang, Thomas Thebaud, Laureano Moro-Velázquez, Dani Byrd, Najim Dehak, and Shrikanth Narayanan. 2025. Vox-profile: A speech foundation model benchmark Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Chri… find
  21. Philip Rose. 2015. Tonation in three chinese wu dialects. In Proc. ICPhS. find
  22. S. Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. 2025. MMAU: A massive multi-task audio understanding and reasoning benchmark. In Proc. ICLR. find
  23. Mingchen Shao, Bingshen Mu, Chengyou Wang, Haizhou Li, Ying Yan, Zhonghua Fu, and Lei Xie. 2025. Towards Building Speech Large Language Models for Multitask Understanding in Low-Resource Languages. CoRR, abs/2509.14804. — [arXiv:2509.14804](https://arxiv.org/abs/2509.14804) find
  24. Zhiyuan Tang, Dong Wang, Yanguang Xu, Jianwei Sun, Xiaoning Lei, Shuaijiang Zhao, Cheng Wen, Xingjun Tan, Chuandong Xie, Shuran Zhou, Rui Yan, Chenjia Lv, Yang Han, Wei Zou, and Xiangang Li. 2021. KeSpeech: An Open Source Speech Dataset of Mandarin and Its Eight Subdialects. In Proc. NeurIPS. find
  25. Dingdong Wang, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. 2025a. MMSU: A massive multi-task spoken language understanding and reasoning benchmark. CoRR, abs/2506.04779. — [arXiv:2506.04779](https://arxiv.org/abs/2506.04779) find
  26. Shuai Wang, Zhaokai Sun, Zhennan Lin, Chengyou Wang, Zhou Pan, and Lei Xie. 2025b. MSU-Bench: Towards Understanding the Conversational Multi-talker Scenarios. CoRR, abs/2508.08155. — [arXiv:2508.08155](https://arxiv.org/abs/2508.08155) find
  27. Shuo Wang, Aishan Maoliniyazi, Xinle Wu, and Xiaofeng Meng. 2020. Emo2vec: Learning emotional embeddings via multi-emotion category. ACM Trans. Internet Techn., 20(2):13:1–13:17. find
  28. Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, Mingrui Chen, Peng Liu, Wang You, Xiangyu Tony Zhang, Xingyuan Li, Xuerui Yang, Yayue Deng, Yechang Huang, Yuxin Li, and 81 others. 2025. Step-audio 2 technical report. CoRR, abs/2507.1663…
  29. Di Wu, Binbin Zhang, Chao Yang, Zhendong Peng, Wenjing Xia, Xiaoyu Chen, and Xin Lei. 2021. U2++: unified two-pass bidirectional end-to-end model for speech recognition. CoRR, abs/2106.05642. — [arXiv:2106.05642](https://arxiv.org/abs/2106.05642) find
  30. Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, and 19 others. 2025a. Qwen3-Omni Technical Report. CoRR, abs/2509.17765.
  31. Kaituo Xu, Feng-Long Xie, Xu Tang, and Yao Hu. 2025b. Fireredasr: Open-source industrial-grade mandarin speech recognition models from encoder-decoder to LLM integration. CoRR, abs/2501.14350. — [arXiv:2501.14350](https://arxiv.org/abs/2501.14350) find
  32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 40 others. 2025. Qwen3 technical report. CoRR, abs/2505.09388.
  33. Yifan Yang, Zheshu Song, Jianheng Zhuo, Mingyu Cui, Jinpeng Li, Bo Yang, Yexing Du, Ziyang Ma, Xunying Liu, Ziyuan Wang, Ke Li, Shuai Fan, Kai Yu, Wei-Qiang Zhang, Guoguo Chen, and Xie Chen. 2024. Gigaspeech 2: An evolving, large-scale and multi-domain ASR corpus for low-resource languages with auto… find
  34. Zhuoyuan Yao, Di Wu, Xiong Wang, Binbin Zhang, Fan Yu, Chao Yang, Zhendong Peng, Xiaoyu Chen, Lei Xie, and Xin Lei. 2021. Wenet: Production oriented streaming and non-streaming end-to-end speech recognition toolkit. In Proc. Interspeech, pages 4054–4058. find
  35. Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J. Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech. In Proc. Interspeech, pages 1526– 1530.
  36. Binbin Zhang, Hang Lv, Pengcheng Guo, Qijie Shao, Chao Yang, Lei Xie, Xin Xu, Hui Bu, Xiaoyu Chen, Chenchen Zeng, Di Wu, and Zhendong Peng. 2022. WENETSPEECH: A 10000+ Hours Multi-Domain Mandarin Corpus for Speech Recognition. In Proc. ICASSP, pages 6182–6186. find
  37. Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, Wenmeng Zhou, and Yingda Chen. 2025. SWIFT: A scalable lightweight infrastructure for fine-tuning. In Proc. AAAI, pages 29733– 29735. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-wenetspeech-wu-datasets-benchmarks-models/