Elamur Research⌕
☰
home / papers / 2026 / Listen, Pause, and Reason: Toward Perception-Grounded Hybrid Reasoning for Audio Understanding

Listen, Pause, and Reason: Toward Perception-Grounded Hybrid Reasoning for Audio Understanding

Paper · 2026 · Jieyi Wang, Yazhe Niu, Dexuan Xu et al.

Summary

Large Audio Language Models often answer audio questions incorrectly because they misperceive the input. They confuse background noise with speech, mix up speakers, or reason from transcripts without using acoustic evidence. Drawing on Auditory Scene Analysis, the authors break perception into layers. They first build PAQA, a multiple-choice Audio-QA dataset whose annotations separate speech from environmental sound and attribute utterances to individual speakers. The annotations include reflection steps. On top of this data they propose HyPeR, a two-stage framework:

  • Stage I: supervised fine-tuning on PAQA teaches explicit perceptual decomposition.
  • Stage II: GRPO refines reasoning. It adds <PAUSE> tokens for silent latent computation, triggered when a sliding-window confidence signal drops on hard acoustic attributes. The reward combines correctness, consistency with the audio evidence, and format.

The authors report clear gains over the base model, with results comparable to much larger systems, especially for noisy and multi-speaker audio. The work suggests that grounding reasoning in structured perception, rather than adding more reasoning, is what makes audio understanding robust.

Key points

  • PAQA dataset:
    • Contains 7,470 multiple-choice items covering two disambiguation levels: Speech vs. Environment and Speaker vs. Speaker.
    • Noisy scenes are built by mixing speech with MUSAN and FSD50K sounds at SNR between 0 and 20 dB.
  • Quote-Presence Test (QPT): measures fuzzy alignment between speaker-attributed text and Qwen3-ASR output. Items scoring below 0.85 are discarded.
  • Reward ablation on the PAQA test set:
    • Full reward: 68.4 accuracy and 91.2 consistency.
    • Without the consistency reward: 64.2 / 78.5.
    • Without length shaping: 67.1 / 89.4.
  • PAUSE results:
    • PAUSE-based latent reasoning is reported to give a large gain on MMAR (+25.5).
    • Hidden-state analysis shows the states keep changing and move steadily closer to the answer representation.
  • Limitations:
    • PAUSE increases training and inference latency.
    • One reflection round helps, but extra rounds lead to "overthinking."
    • The method is weaker on some broader benchmarks.
    • PAQA is limited in scale and domain coverage.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations🎓Fudan UniversityShanghai · China167 works · 20,987 citations🎓Chinese University of Hong KongHong Kong · China126 works · 14,073 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China117 works · 11,281 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksSpeech & spoken language · 480 works

Cite

Cite as Jieyi Wang, Yazhe Niu, Dexuan Xu and Zhongyu Wei. 2026. Listen, Pause, and Reason: Toward Perception-Grounded Hybrid Reasoning for Audio Understanding. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1776

@inproceedings{wang2026listen,
  title = {{Listen, Pause, and Reason: Toward Perception-Grounded Hybrid Reasoning for Audio Understanding}},
  author = {Jieyi Wang, Yazhe Niu, Dexuan Xu and Zhongyu Wei},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1776},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1776}
}

References

52 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Daman Arora and Andrea Zanette. 2025. Training language models to reason efficiently. arXiv preprint arXiv:2502.04463. find
  2. Andrea Banino, Samuel Ritter, and 1 others. 2021. Pondernet: Learning to ponder. In ICML. find
  3. Albert S Bregman. 1994. Auditory scene analysis: The perceptual organization of sound. MIT press. find
  4. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. Qwen2-audio technical report.
  5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  6. Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. 2023. Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  7. Benjamin Elizalde, Soham Deshmukh, and Huaming Wang. 2024. Natural language supervision for general-purpose audio representations. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 336–340. IEEE. find
  8. Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. 2025. Video-r1: Reinforcing video reasoning in mllms. Preprint, arXiv:2503.21776. — [arXiv:2503.21776](https://arxiv.org/abs/2503.21776) find
  9. Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra. 2021. Fsd50k: an open dataset of human-labeled sound events. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:829–852. find
  10. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. 2025. Deep think with confidence. Preprint, arXiv:2508.15260. find
  11. Sreyan Ghosh, Sonal Kumar, Chandra Kiran Reddy Evuru, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. 2025. Reclap: Improving zero shot audio classification by describing sounds. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEE… find
  12. Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. 2024. Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities. arXiv preprint arXiv:2406.11768.
  13. Arushi Goel, Sreyan Ghosh, Jaehyeon Kim, Sonal Kumar, Zhifeng Kong, Sang gil Lee, Chao-Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, and Bryan Catanzaro. 2025. Audio flamingo 3: Advancing audio intelligence with fully open large audio language models. Preprint, arXiv:2507.08128. — [arXiv:2507.08128](https://arxiv.org/abs/2507.08128) find
  14. Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. 2024. Think before you speak: Training language models with pause tokens. Preprint, arXiv:2310.02226. find
  15. Alex Graves. 2017. Adaptive computation time for recurrent neural networks. Preprint, arXiv:1603.08983. find
  16. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseekr1: Incentivizing reasoning capability in llms via reinforcement learning. find
  17. Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models. Preprint, arXiv:2503.06749. find
  18. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720.
  19. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. Preprint, arXiv:2503.09516.
  20. Koray Kavukcuoglu. 2025. Gemini 2.5: Our most intelligent ai model. https://blog.google/technology/google-deepmind/ gemini-model-thinking-updates-march-2025/. Accessed 2025-12-22. ↗
  21. Qiuqiang Kong, Ke Chen, Haohe Liu, Xingjian Du, Taylor Berg-Kirkpatrick, Shlomo Dubnov, and Mark D Plumbley. 2023. Universal source separation with weakly labelled data. arXiv preprint arXiv:2305.07447. — [arXiv:2305.07447](https://arxiv.org/abs/2305.07447) find
  22. Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, and Bryan Catanzaro. 2024. Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities. Preprint, arXiv:2402.01831. — [arXiv:2402.01831](https://arxiv.org/abs/2402.01831) find
  23. Chun-Yi Kuan and Hung-yi Lee. 2025. Can large audio-language models truly hear? tackling hallucinations with multi-task assessment and stepwise audio reasoning. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find
  24. Gang Li, Jizhong Liu, Heinrich Dinkel, Yadong Niu, Junbo Zhang, and Jian Luan. 2025. Reinforcement learning outperforms supervised fine-tuning: A case study on audio question answering. arXiv preprint arXiv:2503.11197. — [arXiv:2503.11197](https://arxiv.org/abs/2503.11197) find
  25. Ziyang Ma, Zhuo Chen, Yuping Wang, Eng Siong Chng, and Xie Chen. 2025a. Audio-cot: Exploring chain-of-thought reasoning in large audio language model. Preprint, arXiv:2501.07246. — [arXiv:2501.07246](https://arxiv.org/abs/2501.07246) find
  26. Ziyang Ma, Yinghao Ma, Yanqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, Kai Li, Keliang Li, Siyou Li, Xinfeng Li, Xiquan Li, Zheng Lian, Yuzhe Liang, Minghao Liu, Zhikang Niu, and 15 others. 2025b. Mmar: A challenging benchmark for deep reasoning in sp… — [arXiv:2505.13032](https://arxiv.org/abs/2505.13032) find
  27. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with… find
  28. Daniel Michelsanti, Zheng-Hua Tan, Shi-Xiong Zhang, Yong Xu, Meng Yu, Dong Yu, and Jesper Jensen. 2021. An overview of deep-learning-based audio-visual speech enhancement and separation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:1368–1396. find
  29. Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, Masahiro Yasuda, Shunsuke Tsubaki, and Keisuke Imoto. 2024. M2d-clap: Masked modeling duo meets clap for learning general-purpose audio-language representation. arXiv preprint arXiv:2406.02032. — [arXiv:2406.02032](https://arxiv.org/abs/2406.02032) find
  30. Yazhe Niu, Yuan Pu, Dongxing Shi, Yudong Lu, Yingtong Xiong, Ruijun Ge, Jiaxuan Sun, Zunian Wan, Shaoang Zhang, and 1 others. 2025. Lightrft. https://github.com/opendilab/LightRFT. ↗
  31. OpenAI. Gpt-4o audio model (gpt-4o-audio-preview) \| openai api documentation. https://platform.openai.com/docs/models/ gpt-4o-audio-preview. Accessed 2025-12-22. ↗
  32. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. MELD: A multimodal multi-party dataset for emotion recognition in conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 527– 536, Flore…
  33. S Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. 2024. Mmau: A massive multi-task audio understanding and reasoning benchmark. Preprint, arXiv:2410.19168. find
  34. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  35. Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, and 1 others. 2026. Qwen3-asr technical report. arXiv preprint arXiv:2601.21337.
  36. Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS).
  37. David Snyder, Guoguo Chen, and Daniel Povey. 2015. Musan: A music, speech, and noise corpus. Preprint, arXiv:1510.08484. — [arXiv:1510.08484](https://arxiv.org/abs/1510.08484) find
  38. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2024. Salmonn: Towards generic hearing abilities for large language models. Preprint, arXiv:2310.13289.
  39. Changhan Wang, Anne Wu, and Juan Pino. 2020. Covost 2 and massively multilingual speech-to-text translation. Preprint, arXiv:2007.10310. find
  40. Dingdong Wang, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. 2025. MMSU: A massive multi-task spoken language understanding and reasoning benchmark. arXiv preprint arXiv:2506.04779. — [arXiv:2506.04779](https://arxiv.org/abs/2506.04779) find
  41. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. Preprint, arXiv:2203.11171.
  42. Hao Zhong, Muzhi Zhu, Zongze Du, Zheng Huang, Canyu Zhao, Mingyu Liu, Wen Wang, Hao Chen, and Chunhua Shen. 2025. Omni-r1: Reinforcement learning for omnimodal reasoning via two-system collaboration. Preprint, arXiv:2505.20256. — [arXiv:2505.20256](https://arxiv.org/abs/2505.20256) find
  43. Cheng Wen, Tingwei Guo, Shuaijiang Zhao, Wei Zou, and Xiangang Li. 2025. Sari: Structured audio reasoning via curriculum-guided reinforcement learning. Preprint, arXiv:2504.15900. — [arXiv:2504.15900](https://arxiv.org/abs/2504.15900) find
  44. Shu Wu, Chenxing Li, Wenfu Wang, Hao Zhang, Hualei Wang, Meng Yu, and Dong Yu. 2025. Audio-thinker: Guiding audio language model when and how to think via reinforcement learning. Preprint, arXiv:2508.08039. — [arXiv:2508.08039](https://arxiv.org/abs/2508.08039) find
  45. Zhifei Xie, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, and Chunyan Miao. 2025. Audio-reasoner: Improving reasoning capability in large audio language models. Preprint, arXiv:2503.02318.
  46. Zhen Xiong, Yujun Cai, Zhecheng Li, Junsong Yuan, and Yiwei Wang. 2025. Thinking with sound: Audio chain-of-thought enables multimodal reasoning in large audio-language models. arXiv preprint arXiv:2509.21749.
  47. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, and 1 others. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215. — Qwen2.5-Omni Technical Report
  48. Xuenan Xu, Heinrich Dinkel, Mengyue Wu, and Kai Yu. 2021. Text-to-audio grounding: Building correspondence between captions and sound events. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 606–610. IEEE. find
  49. Rongkun Xue, Yazhe Niu, Shuai Hu, Zixin Yin, Yongqiang Yao, and Jing Yang. 2025. Hh-codec: High compression high-fidelity discrete neural codec for spoken language modeling. Preprint, arXiv:2507.18897. — [arXiv:2507.18897](https://arxiv.org/abs/2507.18897) find
  50. Pinci Yang, Xin Wang, Xuguang Duan, Hong Chen, Runze Hou, Cong Jin, and Wenwu Zhu. 2022. Avqa: A dataset for audio-visual question answering on videos. In Proceedings of the 30th ACM International Conference on Multimedia, pages 3480–3491. find
  51. Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, Ligeng Zhu, Yuanhang Su, Sean Lin, An-Chieh Cheng, Zhen Wan, Jinchuan Tian, Yuming Lou, Dong Yang, Zhijian Liu, Yukang Chen, Ambrish Dantrey, Ehsan Jahangiri, Sreyan Ghosh, Daguang Xu, Ehsan Hosseini-Asl, and 13 others. 2025. Omnivinci: Enhanci… find
  52. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-listen-pause-reason-perception-grounded/