Elamur Research⌕
☰
home / papers / 2026 / Chronological Thinking in Full-Duplex Spoken Dialogue Language Models

Chronological Thinking in Full-Duplex Spoken Dialogue Language Models

Summary

Full-duplex spoken dialogue language models (SDLMs) listen and speak at the same time. While the user talks, though, current systems just emit a stream of silence tokens. This wastes the listening period and may push an autoregressive model toward degenerate repetition. The authors propose CT-SDLM (implemented as CT-Duplex), which fills that period with Chronological Thinking: a compact chain of structured reasoning nodes produced step by step as speech arrives.

The node types draw on the ACT-R cognitive architecture. Thinking is strictly causal, using only audio heard so far, and can be cut off at any point. The agent therefore starts replying as soon as the user stops, with no added delay.

The architecture has four parts:

  • a streaming speech encoder
  • an LLM backbone that predicts only text tokens
  • a separate Transformer decoder that produces speech tokens
  • an audio codec

Thinking improves response quality, especially on reasoning-heavy task-oriented dialogue, and helps little on factual QA. Turn-taking and barge-in performance stay competitive. The work suggests idle listening time can be turned into useful computation in real-time voice agents.

Key points

  • The thinking chain uses five node types mapped to ACT-R modules and replaces the <SIL> tokens. The user speech encoder runs at 12.5 Hz.
  • On SpokenWOZ, thinking gives an 8.75% improvement. Gains on MtBenchEval multi-turn dialogue are modest, and on Llama Questions and Web Questions they are negligible.
  • The 1.7B model beats larger full-duplex baselines such as Moshi (7B) on factual QA. GLM-4-Voice, a half-duplex 9B model, scores higher.
  • On the Impatient dataset, barge-in success rises from 88.63% to 94.05% with thinking, and barge-in latency is roughly unchanged (0.53 → 0.54). Turn-taking latency increases from 0.45 to 0.68, which the authors still describe as comparable.
  • In an A/B test with 10 raters on SpokenWOZ, the thinking variant was preferred for both audio fidelity and content quality, with win rates of about 55–64%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.sigdial-1.33

@inproceedings{wu2026chronological,
  title = {{Chronological Thinking in Full-Duplex Spoken Dialogue Language Models}},
  author = {Wu, Donghang and Zhang, Haoyang and Chen, Chen and Zhang, Tianyu and Tian, Fei and Yang, Xuerui and Yu, Gang and Liu, Hexin and Hou, Nana and Hu, Yuchen and Chng, Eng Siong},
  booktitle = {Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2026},
  doi = {10.18653/v1/2026.sigdial-1.33},
  url = {https://doi.org/10.18653/v1/2026.sigdial-1.33}
}

References

55 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chen Chen, Ke Hu, Chao-Han Huck Yang, Ankita Pasad, Edresson Casanova, Weiqing Wang, Szu-Wei Fu, Jason Li, Zhehuai Chen, Jagadeesh Balam, and 1 others. Reinforcement learning enhanced full-duplex spoken dialogue language models for conversational interactions. In Second Conference on Language Modeli… find
  2. Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, and Colin Raffel. 2019. Monotonic infinite lookback attention for simultaneous machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pa… find
  3. Chen Chen, Ke Hu, Chao-Han Huck Yang, Ankita Pasad, Edresson Casanova, Weiqing Wang, Szu-Wei Fu, Jason Li, Zhehuai Chen, Jagadeesh Balam, and 1 others. 2025a. Reinforcement learning enhanced full-duplex spoken dialogue language models for conversational interactions. In Second Conference on Language… find
  4. Siddhant Arora, Jinchuan Tian, Hayato Futami, Jiatong Shi, Yosuke Kashiwagi, Emiru Tsunoo, and Shinji Watanabe. 2025. Chain-of-thought reasoning in streaming full-duplex end-to-end spoken dialogue systems. arXiv preprint arXiv:2510.02066. — [arXiv:2510.02066](https://arxiv.org/abs/2510.02066) find
  5. Junjie Chen, Yao Hu, Junjie Li, Kangyue Li, Kun Liu, Wenpeng Li, Xu Li, Ziyuan Li, Feiyu Shen, Xu Tang, and 1 others. 2025b. Fireredchat: A pluggable, full-duplex voice interaction system with cascaded and semi-cascaded implementations. arXiv preprint arXiv:2509.06502. — [arXiv:2509.06502](https://arxiv.org/abs/2509.06502) find
  6. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1533–1544. find
  7. Stefan Brachmanski. Subjective assessment of quality of audio and video signals by means of ab test. Qian Chen, Yafeng Chen, Yanni Chen, Mengzhe Chen, Yingda Chen, Chong Deng, Zhihao Du, Ruize Gao, Changfeng Gao, Zhifu Gao, and 1 others. 2025c. Minmo: A multimodal large language model for seamless v… — [arXiv:2501.06282](https://arxiv.org/abs/2501.06282) find
  8. Cheng-Han Chiang, Xiaofei Wang, Linjie Li, Chung-Ching Lin, Kevin Lin, Shujie Liu, Zhendong Wang, Zhengyuan Yang, Hung-yi Lee, and Lijuan Wang. 2025. Shanks: Simultaneous hearing and thinking for spoken language models. arXiv preprint arXiv:2510.06917. find
  9. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. find
  10. Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. 2024. Medusa: Simple llm inference acceleration framework with multiple decoding heads. In International Conference on Machine Learning, pages 5209–5235. PMLR. find our card
  11. Francesco Calimeri, Giovambattista Ianni, Francesco Pacenza, Simona Perri, and Jessica Zangari. 2021. Stream reasoning with incremental grounding. In 5th Stream Reasoning Workshop. find
  12. Edresson Casanova, Ryan Langman, Paarth Neekhara, Shehzeen Hussain, Jason Li, Subhankar Ghosh, Ante Jukić, and Sang-gil Lee. 2025a. Low frame-rate speech codec: a codec designed for fast high-quality speech llm training and inference. In ICASSP 2025-2025 IEEE International Conference on Acoustics, S… find
  13. Yuya Chiba and Ryuichiro Higashinaka. 2025. Investigating the impact of incremental processing and voice activity projection on spoken dialogue systems. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3687–3696. find
  14. Christopher Cieri, David Miller, and Kevin Walker. 2004. The fisher corpus: A resource for the next generations of speech-to-text. In LREC, volume 4, pages 69–71. find our card
  15. Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, Ziqiao Meng, Guangyan Zhang, Qichao Wang, Yiwen Guo, and Irwin King. 2024. Recent advances in speech language models: A survey. arXiv preprint arXiv:2410.03751. find
  16. Fahim Dalvi, Nadir Durrani, Hassan Sajjad, and Stephan Vogel. 2018. Incremental decoding and training methods for simultaneous translation in neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua… find
  17. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037. find our card
  18. Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, and 1 others. 2025. Kimi-audio technical report. arXiv preprint arXiv:2504.18425. find
  19. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764–10799. PMLR. find
  20. Jian Guan and Minlie Huang. 2023. Mitigating the learning bias towards repetition by self-contrastive training for open-ended generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6897–6909. find
  21. Ting-En Lin, Yuchuan Wu, Fei Huang, Luo Si, Jian Sun, and Yongbin Li. 2022. Duplex conversation: Towards human-like interaction in spoken dialogue systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3299–3308. find
  22. Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, and 1 others. 2019. Stacl: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. In Proceedings of the 57th Annual… find our card
  23. Xutai Ma, Juan Miguel Pino, James Cross, Liezl Puzon, and Jiatao Gu. 2020. Monotonic multihead attention. In International Conference on Learning Representations. find
  24. Ziyang Ma, Yakun Song, Chenpeng Du, Jian Cong, Zhuo Chen, Yuping Wang, Yuxuan Wang, and Xie Chen. 2025. Language model can listen while speaking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24831–24839. find
  25. Yanzhang He, Tara N Sainath, Rohit Prabhavalkar, Ian McGraw, Raziel Alvarez, Ding Zhao, David Rybach, Anjuli Kannan, Yonghui Wu, Ruoming Pang, and 1 others. 2019. Streaming end-to-end speech recognition for mobile devices. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Si… find
  26. Niko Moritz, Takaaki Hori, and Jonathan Le. 2020. Streaming automatic speech recognition with the transformer model. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6074–6078. IEEE. find
  27. Ke Hu, Ehsan Hosseini-Asl, Chen Chen, Edresson Casanova, Subhankar Ghosh, Piotr Żelasko, Zhehuai Chen, Jason Li, Jagadeesh Balam, and Boris Ginsburg. 2025. Efficient and direct duplex modeling for speech-to-speech language model. arXiv preprint arXiv:2505.15670. — [arXiv:2505.15670](https://arxiv.org/abs/2505.15670) find
  28. Eliya Nachmani, Alon Levkovitch, Roy Hirsch, Julian Salazar, Chulayuth Asawaroengchai, Soroosh Mariooryad, Ehud Rivlin, RJ Skerry-Ryan, and Michelle Tadmor Ramanovich. 2023. Spoken question answering and speech continuation using spectrogram-powered llm. arXiv preprint arXiv:2305.15255. — [arXiv:2305.15255](https://arxiv.org/abs/2305.15255) find
  29. Ailin Huang, Boyong Wu, Bruce Wang, Chao Yan, Chen Hu, Chengli Feng, Fei Tian, Feiyu Shen, Jingbei Li, Mingrui Chen, and 1 others. 2025. Step-audio: Unified understanding and generation in intelligent speech interaction. arXiv preprint arXiv:2502.11946. find
  30. Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoit Sagot, Abdelrahman Mohamed, and 1 others. 2023. Generative spoken dialogue language modeling. Transactions of the Association for Computational Linguistics, 11:250–266. find
  31. Hatim Khouzaimi, Romain Laroche, and Fabrice Lefèvre. 2016. Reinforcement learning for turn-taking management in incremental spoken dialogue systems. In IJCAI, pages 2831–2837. find
  32. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. find our card
  33. Oleksii Kuchaiev, Jason Li, Huyen Nguyen, Oleksii Hrinchuk, Ryan Leary, Boris Ginsburg, Samuel Kriman, Stanislav Beliaev, Vitaly Lavrukhin, Jack Cook, and 1 others. 2019. Nemo: a toolkit for building ai applications using neural modules. arXiv preprint arXiv:1909.09577. find
  34. Borui Liao, Yulong Xu, Jiao Ou, Kaiyuan Yang, Weihua Jian, Pengfei Wan, and Di Zhang. 2025. Flex-duo: A pluggable system for enabling full-duplex capabilities in speech dialogue systems. arXiv preprint arXiv:2502.13472. — [arXiv:2502.13472](https://arxiv.org/abs/2502.13472) find
  35. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. arXiv preprint. find our card
  36. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084. find
  37. Frank E Ritter, Farnaz Tehranchi, and Jacob D Oury. 2019. Act-r: A cognitive architecture for modeling cognition. Wiley Interdisciplinary Reviews: Cognitive Science, 10(3):e1488. find
  38. Ruhi Sarikaya, Yuqing Gao, Hakan Erdogan, and Michael Picheny. 2002. Turn-based language modeling for spoken dialog systems. In 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages I–781. IEEE. find
  39. Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, Huayang Li, and Jian Li. 2022. Learning to break the loop: Analyzing and mitigating repetitions for neural text generation. Advances in Neural Information Processing Systems, 35:3082–3095. find
  40. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551. find our card
  41. Kenta Yamamoto, Ryu Takeda, and Kazunori Komatani. 2025. Analysis of voice activity detection errors in api-based streaming asr for human-robot dialogue. In Proceedings of the 15th International Workshop on Spoken Dialogue Systems Technology, pages 245– 253. find
  42. David Schlangen and Gabriel Skantze. 2011. A general, abstract model of incremental dialogue processing. Dialogue & Discourse, 2(1):83–111. find our card
  43. Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, and Xie Chen. 2025. Uro-bench: A comprehensive benchmark for end-to-end spoken dialogue models. arXiv preprint arXiv:2502.17810. — [arXiv:2502.17810](https://arxiv.org/abs/2502.17810) find
  44. Shuzheng Si, Wentao Ma, Haoyu Gao, Yuchuan Wu, Ting-En Lin, Yinpei Dai, Hangyu Li, Rui Yan, Fei Huang, and Yongbin Li. 2023. Spokenwoz: A large-scale speech-text benchmark for spoken task-oriented dialogue agents. Advances in Neural Information Processing Systems, 36:39088–39118. find
  45. RJ Skerry-Ryan, Eric Battenberg, Ying Xiao, Yuxuan Wang, Daisy Stanton, Joel Shor, Ron Weiss, Rob Clark, and Rif A Saurous. 2018. Towards end-to-end prosody transfer for expressive speech synthesis with tacotron. In international conference on machine learning, pages 4693–4702. PMLR. find
  46. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  47. Bandhav Veluri, Benjamin N Peloquin, Bokai Yu, Hongyu Gong, and Shyamnath Gollakota. 2024. Beyond turn-based interfaces: Synchronous llms as full-duplex dialogue agents. arXiv preprint arXiv:2409.15594. find
  48. Peng Wang, Songshuo Lu, Yaohua Tang, Sijie Yan, Wei Xia, and Yuanjun Xiong. 2024. A full-duplex speech dialogue scheme based on large language model. Advances in Neural Information Processing Systems, 37:13372–13403. find
  49. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations. find our card
  50. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837. find our card
  51. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. find
  52. Wenyi Yu, Siyin Wang, Xiaoyu Yang, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Guangzhi Sun, Lu Lu, Yuxuan Wang, and Chao Zhang. 2024. Salmonn-omni: A codec-free llm for full-duplex speech understanding and generation. arXiv preprint arXiv:2411.18138. — [arXiv:2411.18138](https://arxiv.org/abs/2411.18138) find
  53. Aohan Zeng, Zhengxiao Du, Mingdao Liu, Kedong Wang, Shengmin Jiang, Lei Zhao, Yuxiao Dong, and Jie Tang. 2024. Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot. arXiv preprint arXiv:2412.02612. find
  54. Hao Zhang, Weiwei Li, Rilin Chen, Vinay Kothapally, Meng Yu, and Dong Yu. 2025. Llm-enhanced dialogue management for full-duplex spoken dialogue systems. arXiv preprint arXiv:2502.14145. — [arXiv:2502.14145](https://arxiv.org/abs/2502.14145) find
  55. Wenhong Zhu, Hongkun Hao, and Rui Wang. 2023. Penalty decoding: Well suppress the self-reinforcement effect in open-ended text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1218–1228. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/wu-2026-chronological-thinking-full-duplex-spoken/