Elamur Research⌕
☰
home / papers / 2026 / Reproducing Proficiency-Conditioned Dialogue Features with Full-duplex Spoken Dialogue Models

Reproducing Proficiency-Conditioned Dialogue Features with Full-duplex Spoken Dialogue Models

Paper · 2026 · Takao Obi, Sadahiro Yoshikawa, Mao Saeki et al.

Summary

Full-duplex spoken dialogue models can listen and speak simultaneously, but there is little evidence on whether they reproduce human conversational behaviour. The authors test this using second-language (L2) learner interviews. They fine-tuned Moshi on about 24k interview dialogues collected by the InteLLA assessment system and trained one separate model for each aggregated CEFR level (A, B and C). To prevent long silences at inference time, training used only segments that run from the end of the interviewer's turn to the end of the learner's turn. Each level model then completed live WebRTC interviews with InteLLA. The authors measured reaction time, response frequency and seven fluency metrics, and compared the results against matched samples of real learner interviews. The level-conditioned models mostly preserved the ordering of proficiency levels found in human data. Their distributions, however, matched human ones only in part. The work suggests that full-duplex models could serve as proficiency-aware simulated users, which would reduce the human effort needed to test and evaluate spoken assessment systems.

Key points

  • Training data and setup: 24,320 InteLLA interviews (2,451 h raw, 1,936 h after segment extraction), heavily skewed toward level B. Training used the public moshi-finetune recipe for 3 epochs on 8 NVIDIA H200 GPUs.
  • Evaluation sessions: 400 sessions per level model (four single-topic scenarios, 100 each), run with sampling temperature 0.95 and Agora RTC audio streaming.
  • Metrics:
    • Fluency was annotated automatically with the model of Matsuura et al. (2025), covering AR, MCPR, ECPR, PR, DR, SR and MLR.
    • Statistical analysis used Kruskal-Wallis tests, Holm-corrected Mann–Whitney U tests, Cliff's delta, and Wasserstein distance with bootstrap CIs.
  • Level differences: Kruskal-Wallis tests found significant differences between levels in both human and model data, except for the models' response frequency.
  • Distributional alignment: Mean and median gaps between models and humans sometimes fell within the range of human variability, but Wasserstein distances generally did not.
  • Limitations noted by the authors: class imbalance, heuristic training and decoding choices, and no measurement of overlap or backchannel behaviour.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 624 works

Cite

Cite as Takao Obi, Sadahiro Yoshikawa, Mao Saeki, Masaki Eguchi and Yoichi Matsuyama. 2026. Reproducing Proficiency-Conditioned Dialogue Features with Full-duplex Spoken Dialogue Models. Proceedings of the 16th International Workshop on Spoken Dialogue System Technology.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{obi2026reproducing,
  title = {{Reproducing Proficiency-Conditioned Dialogue Features with Full-duplex Spoken Dialogue Models}},
  author = {Takao Obi, Sadahiro Yoshikawa, Mao Saeki, Masaki Eguchi and Yoichi Matsuyama},
  booktitle = {Proceedings of the 16th International Workshop on Spoken Dialogue System Technology},
  year = {2026},
  url = {https://aclanthology.org/2026.iwsds-1.4/}
}

References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Companion volume Language Policy Programme Education Policy Division Education Department Council of Europe. find
  2. Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. 2023a. Audiolm: a language modeling approach to audio generation.
  3. Atsumoto Ohashi, Shinya Iizuka, Jingjing Jiang, and Ryuichiro Higashinaka. 2025. Towards a japanese full-duplex spoken dialogue system. In Proceedings of INTERSPEECH 2025, pages 1783–1787. find
  4. Harvey Sacks, Emanuel Schegloff, and Gail Jefferson. 1974. A simplest systematics for the organization of turn-taking for conversation. Language, 50:696–735. find
  5. Zalán Borsos, Matt Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. 2023b. Soundstorm: Efficient parallel audio generation. Preprint, arXiv:2305.09636. — [arXiv:2305.09636](https://arxiv.org/abs/2305.09636) find
  6. Mao Saeki, Hiroaki Takatsu, Fuma Kurata, Shungo Suzuki, Masaki Eguchi, Ryuki Matsuura, Kotaro Takizawa, Sadahiro Yoshikawa, and Yoichi Matsuyama. 2024. InteLLA: Intelligent language learning assistant for assessing language proficiency through interviews and roleplays. In Proceedings of SIGDIAL 2024… find
  7. Chen Chen, Ke Hu, Chao-Han Huck Yang, Ankita Pasad, Edresson Casanova, Weiqing Wang, Szu-Wei Fu, Jason Li, Zhehuai Chen, Jagadeesh Balam, and Boris Ginsburg. 2025. Reinforcement learning enhanced full-duplex spoken dialogue language models for conversational interactions. In Second Conference on Lan… find
  8. Gabriel Skantze. 2021. Turn-taking in conversational systems and human-robot interaction: A review. Computer Speech & Language, 67:101178. find
  9. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: a speech-text foundation model for real-time dialogue. Preprint, arXiv:2410.00037.
  10. Marco Tagliasacchi, Yunpeng Li, Karolis Misiunas, and Dominik Roblek. 2020. Seanet: A multimodal speech enhancement network. Preprint, arXiv:2009.02095. — [arXiv:2009.02095](https://arxiv.org/abs/2009.02095) find
  11. Mattias Heldner and Jens Edlund. 2010. Pauses, gaps and overlaps in conversations. Journal of Phonetics, 38:555–568. find
  12. Hiroaki Takatsu, Shungo Suzuki, Masaki Eguchi, Ryuki Matsuura, Mao Saeki, and Yoichi Matsuyama. 2025. Gnowsis: Multimodal multitask learning for oral proficiency assessments. Computer Speech & Language, page 101860. find
  13. Guan-Ting Lin, Shih-Yun Shan Kuan, Qirui Wang, Jiachen Lian, Tingle Li, Shinji Watanabe, and Hung yi Lee. 2025. Full-duplex-bench v1.5: Evaluating overlap handling for full-duplex speech models. Preprint, arXiv:2507.23159. — [arXiv:2507.23159](https://arxiv.org/abs/2507.23159) find
  14. Shravan Vasishth and Andrew Gelman. 2021. How to embrace variation and accept uncertainty in linguistic and psycholinguistic data analysis. Linguistics, 59(5):1311–1342. find
  15. Rao Ma, Mengjie Qian, Siyuan Tang, Stefano Bannò, Kate M. Knill, and Mark J.F. Gales. 2025. Assessment of L2 Oral Proficiency using Speech Large Language Models. In Proceedings of INTERSPEECH 2025, pages 5078–5082. find
  16. Bandhav Veluri, Benjamin N Peloquin, Bokai Yu, Hongyu Gong, and Shyamnath Gollakota. 2024. Beyond turn-based interfaces: Synchronous LLMs as full-duplex dialogue agents. In Proceedings of EMNLP 2024, pages 21390–21402. find
  17. Ryuki Matsuura, Shungo Suzuki, Kotaro Takizawa, Mao Saeki, and Yoichi Matsuyama. 2025. Gauging the validity of machine learning-based temporal feature annotation to measure fluency in speech automatically. Research Methods in Applied Linguistics, 4(1):100177. find
  18. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2023. Neural codec language models are zero-shot text to speech synthesizers. Preprint, arXiv:2301.02111.
  19. Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoît Sagot, Abdelrahman Mohamed, and Emmanuel Dupoux. 2023. Generative spoken dialogue language modeling. Transactions of the Association for Computational Linguistics, 11:250–266.
  20. Peng Wang, Songshuo Lu, Yaohua Tang, Sijie Yan, Wei Xia, and Yuanjun Xiong. 2024. A full-duplex speech dialogue scheme based on large language model. In Proceedings of NeurIPS 2024. find
  21. Anne Wu, Laurent Mazaré, Neil Zeghidour, and Alexandre Défossez. 2025. Aligning spoken dialogue models from user interactions. In Forty-second International Conference on Machine Learning. find
  22. Brian North and Enrica Piccardo. 2020. Companion volume COMMON EUROPEAN FRAMEWORK OF REFERENCE FOR LANGUAGES: LEARN-ING, TEACHING, ASSESSMENT COMMON EURO-PEAN FRAMEWORK OF REFERENCE FOR LAN-GUAGES: LEARNING, TEACHING, ASSESSMENT Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco T…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/obi-2026-reproducing-proficiency-conditioned-dialogue-features/