Elamur Research⌕
☰
home / papers / 2026 / UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment

UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment

Paper · 2026 · Yuanyuan Wang, Dongchao Yang, Yayue Deng et al.

Summary

Speech generation is still judged mostly through human ratings such as MOS, which are costly, subjective and hard to reproduce. Automatic rewards like WER, SIM or UTMOS each capture only one property and give no explanation. Earlier AudioLLM judges cover only narrow settings, such as single utterances or single-turn dialogue. The authors propose UniSRM, a single reward model that breaks speech assessment into several dimensions and writes explicit reasoning before giving its verdict.

To train and test it, they built UniSRM-Data and UniSRM-Bench, which cover four tasks ranging from utterance-level quality to context-level coherence. Training has two stages. Supervised fine-tuning comes first, followed by RCR-GRPO, a reinforcement-learning step that rewards consistency between per-dimension reasoning and the final decision.

UniSRM gives more accurate and human-aligned judgments than its baselines across all benchmark tasks. It also generalizes to external human-rated datasets, including SOMOS, which was unseen in training. This makes it a practical, interpretable candidate reward signal for optimizing speech generation systems.

Key points

  • Four benchmark tasks:
    • pairwise A/B preference between TTS outputs, labeled by Gemini-2.0-Flash on four scored dimensions;
    • single-utterance quality scoring built on QualiSpeech, with seven aspects;
    • scenario-aware style consistency on ESD, with contexts generated by GPT-4.1 and hard negatives;
    • multi-turn dialogue evaluation based on DailyTalk, annotated with Gemini-2.5-Pro.
  • Why RCR-GRPO matters: GRPO with only an accuracy reward sometimes scored below the SFT-only model on individual dimensions. The authors attribute this to reasoning drift and shortcut exploitation, which RCR-GRPO reduces.
  • BVCC results: UniSRM reaches a PCC of 0.4977 and an accuracy of 49.16%, compared with 0.3420 PCC for the best Gemini baseline (Gemini-2.5-Flash).
  • SOMOS results (unseen in training): UniSRM obtains the highest PCC, about 0.23–0.26, on both the Clean and Full subsets. It beats DNSMOS, Qwen2.5-Omni-7B, Gemini-2.5-Flash and Gemini-2.5-Pro.
  • Release and limitations: the checkpoint and data are released at github.com/lavendery/UniSRM. The authors note weak coverage of heavy accents and overlapped speech, and a high compute cost.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Chinese University of Hong KongHong Kong · China126 works · 14,073 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Yuanyuan Wang, Dongchao Yang, Yayue Deng, Zhiyong Wu, Steven Y. Guo, Helen M. Meng and Xixin Wu. 2026. UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.2150

@inproceedings{wang2026unisrm,
  title = {{UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessment}},
  author = {Yuanyuan Wang, Dongchao Yang, Yayue Deng, Zhiyong Wu, Steven Y. Guo, Helen M. Meng and Xixin Wu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2150},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2150}
}

References

50 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  2. Chen Chen, Yuchen Hu, Siyin Wang, Helin Wang, Zhehuai Chen, Chao Zhang, Chao-Han Huck Yang, and Eng Siong Chng. 2025a. Audio large language models can be descriptive speech quality evaluators. arXiv preprint arXiv:2501.17202. — [arXiv:2501.17202](https://arxiv.org/abs/2501.17202) find
  3. Jingyi Chen, Ju Seung Byun, Micha Elsner, Pichao Wang, and Andrew Perrault. 2025b. Fine-tuning text-to-speech diffusion models using reinforcement learning with human feedback. In Proc. find
  4. Interspeech 2025, pages 3454–3458. find
  5. Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, JianZhao JianZhao, Kai Yu, and Xie Chen. 2025c. F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape…
  6. Erica Cooper and Junichi Yamagishi. 2021. How do voices from past speech synthesis challenges compare today? In Proc. find
  7. Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, and 1 others. 2025. Kimi-audio technical report. arXiv preprint arXiv:2504.18425.
  8. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, and 1 others. 2024. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05407. find
  9. Yuchen Hu, Chen Chen, Siyin Wang, Eng Siong Chng, and Chao Zhang. 2024. Robust zero-shot text-to-speech synthesis with reverse inference optimization. arXiv preprint arXiv:2407.02243. — [arXiv:2407.02243](https://arxiv.org/abs/2407.02243) find
  10. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  11. ITU-T. 1996. Recommendation p.800: Methods for subjective determination of transmission quality (08/96). find
  12. Shengpeng Ji, Tianle Liang, Yangzhuo Li, Jialong Zuo, Minghui Fang, Jinzheng He, Yifu Chen, Zhengqing Liu, Ziyue Jiang, Xize Cheng, and 1 others. 2025. Wavreward: Spoken dialogue models with generalist reward evaluators. arXiv preprint arXiv:2505.09558. — [arXiv:2505.09558](https://arxiv.org/abs/2505.09558) find
  13. Yuma Koizumi, Heiga Zen, Shigeki Karita, Yifan Ding, Kohei Yatabe, Nobuyuki Morioka, Michiel Bacchiani, Yu Zhang, Wei Han, and Ankur Bapna. 2023. Libritts-r: A restored multi-speaker text-to-speech corpus. Preprint, arXiv:2305.18802. — [arXiv:2305.18802](https://arxiv.org/abs/2305.18802) find
  14. Keon Lee, Kyumin Park, and Daeyoung Kim. 2022. Dailytalk: Spoken dialogue dataset for conversational text-to-speech. Preprint, arXiv:2207.01063. — [arXiv:2207.01063](https://arxiv.org/abs/2207.01063) find
  15. Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. 2023. Aligning text-to-image models using human feedback. arXiv preprint arXiv:2302.12192. — [arXiv:2302.12192](https://arxiv.org/abs/2302.12192) find
  16. Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia, Xintao Wang, and 1 others. 2025. Improving video generation with human feedback. arXiv preprint arXiv:2501.13918. — [arXiv:2501.13918](https://arxiv.org/abs/2501.13918) find
  17. Potsawee Manakul, Woody Haosheng Gan, Michael J Ryan, Ali Sartaz Khan, Warit Sirichotedumrong, Kunat Pipatanakul, William Held, and Diyi Yang. 2025. Audiojudge: Understanding what works in large audio model based speech evaluation. arXiv preprint arXiv:2507.12705.
  18. Georgia Maniati, Alexandra Vioni, Nikolaos Ellinas, Karolos Nikitaras, Konstantinos Klapsas, June Sig Sung, Gunu Jho, Aimilios Chalamandaris, and Pirros Tsiakoulis. 2022. Somos: The samsung open mos dataset for the evaluation of neural text-to-speech synthesis. find
  19. Interspeech 2022. speech quality assessment dataset with natural language reasoning and descriptions. arXiv preprint arXiv:2503.20290. — [arXiv:2503.20290](https://arxiv.org/abs/2503.20290) find
  20. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  21. Xihuai Wang, Ziyi Zhao, Siyu Ren, Shao Zhang, Song Li, Xiaoyu Li, Ziwen Wang, Lin Qiu, Guanglu Wan, Xuezhi Cao, and 1 others. 2025c. Audio turing test: Benchmarking the human-likeness of large language model-based text-to-speech systems in chinese. arXiv preprint arXiv:2505.11200. — [arXiv:2505.11200](https://arxiv.org/abs/2505.11200) find
  22. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  23. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36:53728–53741.
  24. Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. Utmos: Utokyo-sarulab system for voicemos challenge 2022. find
  25. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. find
  26. Xiaohui Sun, Ruitong Xiao, Jianye Mo, Bowen Wu, Qun Yu, and Baoxun Wang. 2025. F5r-tts: Improving flow-matching based text-to-speech with group relative policy optimization. arXiv preprint arXiv:2504.02407. — [arXiv:2504.02407](https://arxiv.org/abs/2504.02407) find
  27. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  28. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  29. Hui Wang, Jinghua Zhao, Yifan Yang, Shujie Liu, Junyang Chen, Yanzhe Zhang, Shiwan Zhao, Jinyu Li, Jiaming Zhou, Haoqin Sun, and 1 others. 2025a. Speechllm-as-judges: Towards general and interpretable speech quality evaluation. arXiv preprint arXiv:2510.14664.
  30. Siyin Wang, Wenyi Yu, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Lu Lu, Yu Tsao, Junichi Yamagishi, Yuxuan Wang, and Chao Zhang. 2025b. find
  31. Qualispeech: A Yibin Wang, Zhimin Li, Yuhang Zang, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. 2025d. Unified multimodal chain-of-thought reward model through reinforcement fine-tuning. arXiv preprint arXiv:2505.03318. — [arXiv:2505.03318](https://arxiv.org/abs/2505.03318) find
  32. Yibin Wang, Zhiyu Tan, Junyan Wang, Xiaomeng Yang, Cheng Jin, and Hao Li. 2024a. Lift: Leveraging human feedback for text-to-video model alignment. arXiv preprint arXiv:2412.04814. — [arXiv:2412.04814](https://arxiv.org/abs/2412.04814) find
  33. Yuanyuan Wang, Hangting Chen, Dongchao Yang, Weiqin Li, Dan Luo, Guangzhi Li, Shan Yang, Zhiyong Wu, Helen Meng, and Xixin Wu. 2025e. Unisep: Universal target audio separation with language models at scale. In 2025 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE. find
  34. Yuanyuan Wang, Hangting Chen, Dongchao Yang, Zhiyong Wu, and Xixin Wu. 2025f. Audiocomposer: Towards fine-grained audio generation with natural language descriptions. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  35. Yuanyuan Wang, Hangting Chen, Dongchao Yang, Jianwei Yu, Chao Weng, Zhiyong Wu, and Helen Meng. 2024b. Consistent and relevant: Rethink the query embedding in general sound separation. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 961–96… find
  36. Yuanyuan Wang, Dongchao Yang, Yiwen Shao, Hangting Chen, Jiankun Zhao, Zhiyong Wu, Helen Meng, and Xixin Wu. 2026. Dualspeechlm: Towards unified speech understanding and generation via dual speech token modeling with large language models. In Proceedings of the AAAI Conference on Artificial Intellig… find
  37. LLM-Core-Team Xiaomi. 2025. Mimo-audio: Audio language models are few-shot learners.
  38. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, and 1 others. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215. — Qwen2.5-Omni Technical Report
  39. Dongchao Yang, Haohan Guo, Yuanyuan Wang, Rongjie Huang, Xiang Li, Xu Tan, Xixin Wu, and Helen Meng. 2024a. Uniaudio 1.5: Large language model-driven audio codec is a few-shot audio task learner. Advances in Neural Information Processing Systems, 37:56802–56827. find
  40. Dongchao Yang, Rongjie Huang, Yuanyuan Wang, Haohan Guo, Dading Chong, Songxiang Liu, Xixin Wu, and Helen Meng. 2025a. Simplespeech 2: Towards simple and efficient text-to-speech with flow-based scalar latent transformer diffusion models. IEEE Transactions on Audio, Speech and Language Processing. find
  41. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593. find
  42. Dongchao Yang, Songxiang Liu, Haohan Guo, Jiankun Zhao, Yuanyuan Wang, Helin Wang, Zeqian Ju, Xubo Liu, Xueyuan Chen, Xu Tan, and 1 others. 2025b. Almtokenizer: A low-bitrate and semantic-rich audio codec tokenizer for audio language modeling. In International Conference on Machine Learning, pages 7… find
  43. Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Haohan Guo, Xuankai Chang, Jiatong Shi, Jiang Bian, Zhou Zhao, and 1 others. 2024b. Uniaudio: Towards universal audio generation with large language models. In Forty-first International Conference on Machine Learning. find
  44. Dongchao Yang, Yuanyuan Wang, Dading Chong, Songxiang Liu, Xixin Wu, and Helen Meng. 2026. Uniaudio 2.0: A unified audio language model with text-aligned factorized audio tokenization. arXiv preprint arXiv:2602.04683. — [arXiv:2602.04683](https://arxiv.org/abs/2602.04683) find
  45. Dongchao Yang, Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. 2023. Diffsound: Discrete diffusion model for text-to-sound generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:1720–1733. find
  46. Qize Yang, Shimin Yao, Weixuan Chen, Shenghao Fu, Detao Bai, Jiaxing Zhao, Boyuan Sun, Bowen Yin, Xihan Wei, and Jingren Zhou. 2025c. Humanomniv2: From understanding to omni-modal reasoning with context. arXiv preprint arXiv:2506.21277. — [arXiv:2506.21277](https://arxiv.org/abs/2506.21277) find
  47. Dong Zhang, Zhaowei Li, Shimin Li, Xin Zhang, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. 2024. Speechalign: Aligning speech generation to human preferences. Advances in Neural Information Processing Systems, 37:50343–50360. find
  48. Xueyao Zhang, Chaoren Wang, Huan Liao, Ziniu Li, Yuancheng Wang, Li Wang, Dongya Jia, Yuanzhe Chen, Xiulin Li, Zhuo Chen, and 1 others. 2025. Speechjudge: Towards human-level judgment for speech naturalness. arXiv preprint arXiv:2511.07931. — [arXiv:2511.07931](https://arxiv.org/abs/2511.07931) find
  49. Xiangyu Zhao, Shengyuan Ding, Zicheng Zhang, Haian Huang, Maosongcao Maosongcao, Jiaqi Wang, Weiyun Wang, Xinyu Fang, Wenhai Wang, Guangtao Zhai, and 1 others. 2025. Omnialign-v: Towards enhanced alignment of mllms with human preference. In Proceedings of the 63rd Annual Meeting of the Association f… find
  50. Kun Zhou, Berrak Sisman, Rui Liu, and Haizhou Li. 2022. Emotional voice conversion: Theory, databases and esd. Speech Communication, 137:1– 18. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-unisrm-unified-speech-reward-model/