Elamur Research⌕
☰
home / papers / 2026 / Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions

Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions

Summary

Building counseling chatbots needs multi-turn therapist–client conversations, but real sessions are expensive to collect, hard to scale and raise privacy issues. Earlier work mostly rewrote single-turn Q&A or counseling reports into dialogues, which misses the real back-and-forth of a session. The authors propose Interactive Agents, where two GPT-4 instances role-play a session. The client agent receives a role card, designed with professional counselors and drawn from a pool of real clients, so that it keeps a stable personality and set of concerns. The counselor agent follows a three-stage therapeutic model: exploration, then insight, then action. The resulting synthetic corpus, SimPsyDial, was checked for client fidelity and diversity and for therapeutic quality using WAI-O-S, a working-alliance measure. Two 7B open models fine-tuned on SimPsyDial were then compared in pairwise, arena-style tests against existing mental-health dialogue systems, and they came out on top. This suggests that simulated sessions can replace scarce real data while keeping privacy intact.

Key points

  • Simulation setup:
    • The client always opens with "Hello."
    • A session runs up to 50 turns or until the counselor emits an end token, which is longer than a typical real session.
    • Four professional counselors, all holding master's degrees in psychology, helped design the role cards.
  • Evaluation of the simulated sessions:
    • Role cards strongly shape what the client agent says.
    • Simulated clients are about as diverse as real clients.
    • Therapeutic quality was rated with WAI-O-S, a brief form of the Working Alliance Inventory that is scored by outside observers.
  • Fine-tuned models: The models trained on SimPsyDial are simpsybot_D and simpsybot_Q.
    • They received the highest Elo ratings from both an LLM judge and professional counselors.
    • They outperformed baselines that include a model trained on real counseling dialogues.
    • Models trained on real data sometimes produced odd outputs, such as names or remarks that the session time was over.
  • Human annotation and judge agreement:
    • Three counselors each annotated 97 dialogues, giving 291 in total.
    • As an automatic judge, GPT-4o (gpt-4o-2024-08-06) agreed with the counselors 64.4% of the time, more often than Qwen1.5-110B-Chat did.
  • Release and ethics:
    • Code, data and models are available at github.com/qiuhuachuan/interactive-agents.
    • The study was approved by the Westlake University ethics committee.
    • The authors stress that such systems should not replace professional care without supervision.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.starsem-conference.29

@inproceedings{qiu2026interactive,
  title = {{Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions}},
  author = {Qiu, Huachuan and Lan, Zhenzhong},
  booktitle = {Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.starsem-conference.29},
  url = {https://doi.org/10.18653/v1/2026.starsem-conference.29}
}

References

50 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zahra Abbasiantaeb, Yifei Yuan, Evangelos Kanoulas, and Mohammad Aliannejadi. 2024. Let the llms talk: Simulating human-to-human conversational qa via zero-shot llm-to-llm interactions. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, WSDM ’24, page 8–17, New Yo… find
  2. Jafar Afzali, Aleksander Mark Drzewiecki, Krisztian Balog, and Shuo Zhang. 2023. Usersimcrs: a user simulation toolkit for evaluating conversational recommender systems. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 1160–1163. find
  3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… find our card
  4. Krisztian Balog and ChengXiang Zhai. 2024. Tutorial on user simulation for evaluating information access systems on the web. In Companion Proceedings of the ACM on Web Conference 2024, pages 1254–1257. find
  5. Sebastian Peter Bayerl, Gabriel Roccabruna, Shammur Absar Chowdhury, Tommaso Ciulli, Morena Danieli, Korbinian Riedhammer, and Giuseppe Riccardi. 2022. What can speech and language tell us about the working alliance in psychotherapy. In Interspeech 2022, pages 2443 – 2447. find
  6. Nolwenn Bernard and Krisztian Balog. 2024a. Identifying breakdowns in conversational recommender systems using user simulation. In Proceedings of the 6th ACM Conference on Conversational User Interfaces, pages 1–10. find
  7. Nolwenn Bernard and Krisztian Balog. 2024b. Towards a formal characterization of user simulation objectives in conversational information access. In Proceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval, pages 185–193. find
  8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  9. Gordon Dai, Weijia Zhang, Jinhan Li, Siqi Yang, Chidera Onochie lbe, Srihas Rao, Arthur Caetano, and Misha Sra. 2024. Artificial leviathan: Exploring social evolution of llm agents through the lens of hobbesian social contract theory. find
  10. Andrew Darchuk, Victor Wang, David Weibel, Jennifer Fende, Timothy Anderson, and Adam Horvath. 2000. Manual for the working alliance inventory – observer form (wai-o): Revision iv. find
  11. DeepSeek-AI. 2024a. Deepseek llm: Scaling open-source language models with longtermism. find
  12. DeepSeek-AI. 2024b. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. find
  13. Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinka… find our card
  14. Igor Grossmann, Matthew Feinberg, Dawn C Parker, Nicholas A Christakis, Philip E Tetlock, and William A Cunningham. 2023. Ai and the transformation of social science research. Science, 380(6650):1108–1109. find
  15. Clara E Hill. 2020. Helping skills: Facilitating exploration, insight, and action. American Psychological Association. find
  16. Stefan G Hofmann, Anu Asnaani, Imke JJ Vonk, Alice T Sawyer, and Angela Fang. 2012. The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive therapy and research, 36:427–440. find
  17. Bihao Hu, Longwei Zheng, Jiayi Zhu, Lishan Ding, Yilei Wang, and Xiaoqing Gu. 2024. Teaching plan generation and evaluation with gpt-4: Unleashing the potential of llm in instructional design. IEEE Transactions on Learning Technologies, 17:1471–1485. find
  18. Chen Huang, Peixin Qin, Yang Deng, Wenqiang Lei, Jiancheng Lv, and Tat-Seng Chua. 2024. Concept– an evaluation protocol on conversation recommender systems with system-and user-centric factors. arXiv preprint arXiv:2404.03304. — [arXiv:2404.03304](https://arxiv.org/abs/2404.03304) find
  19. Unggi Lee, Jiyeong Bae, Dohee Kim, Sookbun Lee, Jaekwon Park, Taekyung Ahn, Gunho Lee, Damji Stratton, and Hyeoncheol Kim. 2024. Language model can do knowledge tracing: Simple but effective method to integrate language model and knowledge tracing task. find
  20. Unggi Lee, Sanghyeok Lee, Junbo Koh, Yeil Jeong, Haewon Jung, Gyuri Byun, Jewoong Moon, Jieun Lim, and † HyeoncheolKim. 2023. Generative agent for teacher training: Designing educational problem-solving simulations with large language model-based agents for pre-service teachers. In NeurIPS’23 Worksh… find
  21. Anqi Li, Lizhi Ma, Yaling Mei, Hongliang He, Shuai Zhang, Huachuan Qiu, and Zhenzhong Lan. 2023a. Understanding client reactions in online mental health counseling. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10358– 10376,… find
  22. Han Li, Renwen Zhang, Yi-Chieh Lee, Robert E Kraut, and David C Mohr. 2023b. Systematic review and meta-analysis of ai-based conversational agents for promoting mental health and well-being. NPJ Digital Medicine, 6(1):236. find
  23. Junkai Li, Siyu Wang, Meng Zhang, Weitao Li, Yunghwei Lai, Xinhui Kang, Weizhi Ma, and Yang Liu. 2024. Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv preprint arXiv:2405.02957. — [arXiv:2405.02957](https://arxiv.org/abs/2405.02957) find
  24. June M Liu, Donghao Li, He Cao, Tianhe Ren, Zeyi Liao, and Jiamin Wu. 2023. Chatcounselor: A large language models for mental health support. arXiv preprint arXiv:2309.15461. — [arXiv:2309.15461](https://arxiv.org/abs/2309.15461) find
  25. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. 2024. The ai scientist: Towards fully automated open-ended scientific discovery. find our card
  26. OpenAI. 2024. Gpt-4 technical report. of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, New York, NY, USA. Association for Computing Machinery. find our card
  27. Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. Social simulacra: Creating populated prototypes for social computing systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22, New Yo… find
  28. William E Piper. 2008. Underutilization of short-term group therapy: Enigmatic or understandable? Psychotherapy Research, 18(2):127–138. find
  29. Huachuan Qiu, Hongliang He, Shuai Zhang, Anqi Li, and Zhenzhong Lan. 2024a. SMILE: Single-turn to multi-turn inclusive language expansion via ChatGPT for mental health support. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 615–636, Miami, Florida, USA. Association f… find
  30. Huachuan Qiu, Anqi Li, Lizhi Ma, and Zhenzhong Lan. 2024b. Psychat: A client-centric dialogue system for mental health support. In 2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pages 2979–2984. find
  31. Huachuan Qiu, Tong Zhao, Anqi Li, Shuai Zhang, Hongliang He, and Zhenzhong Lan. 2023. A benchmark for understanding dialogue safety in mental health support. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 1–13. Springer. find
  32. Carl R Rogers. 1946. Significant aspects of client-centered therapy. American psychologist, 1(10):415– 422. find
  33. Samuel Schmidgall, Rojin Ziaei, Carl Harris, Eduardo Reis, Jeffrey Jopling, and Michael Moor. 2024. Agentclinic: a multimodal agent benchmark to evaluate ai in simulated clinical environments. arXiv preprint arXiv:2405.07960. — [arXiv:2405.07960](https://arxiv.org/abs/2405.07960) find
  34. Hao Sun, Zhenru Lin, Chujie Zheng, Siyang Liu, and Minlie Huang. 2021. Psyqa: A chinese dataset for generating long counseling text for mental health support. arXiv preprint arXiv:2106.01702. find
  35. Shangqing Tu, Zheyuan Zhang, Jifan Yu, Chunyang Li, Siyu Zhang, Zijun Yao, Lei Hou, and Juanzi Li. 2023. Littlemu: Deploying an online virtual teaching assistant via heterogeneous sources integration and chain of teach prompts. In Proceedings of the 32nd ACM International Conference on Information a… find
  36. Paul Owoicho, Ivan Sekulic, Mohammad Aliannejadi, Jeffrey Dalton, and Fabio Crestani. 2023. Exploiting simulated user feedback for conversational search: Ranking, rewriting, and beyond. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval… find
  37. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra Bruce E Wampold. 2013. The great psychotherapy debate: Models, methods, and findings. Routledge. find
  38. Jiashuo Wang, Yang Xiao, Yanran Li, Changhe Song, Chunpu Xu, Chenhao Tan, and Wenjie Li. 2024a. Towards a client-centered assessment of llm therapists by client simulation. arXiv preprint arXiv:2406.12266. — [arXiv:2406.12266](https://arxiv.org/abs/2406.12266) find
  39. Ruiyi Wang, Stephanie Milani, Jamie C. Chiu, Jiayin Zhi, Shaun M. Eack, Travis Labrum, Samuel M Murphy, Nev Jones, Kate V Hardy, Hong Shen, Fei Fang, and Zhiyu Chen. 2024b. PATIENT-ψ: Using large language models to simulate patients for training mental health professionals. In Proceedings of the 202… find
  40. C Seth Warren. 1998. Models of brief psychodynamic therapy: A comparative approach. Psychology. find
  41. Joseph Weizenbaum. 1966. Eliza—a computer program for the study of natural language communication between man and machine. Commun. ACM, 9(1):36–45. find
  42. Chengxing Xie, Canyu Chen, Feiran Jia, Ziyu Ye, Shiyang Lai, Kai Shu, Jindong Gu, Adel Bibi, Ziniu Hu, David Jurgens, James Evans, Philip Torr, Bernard Ghanem, and Guohao Li. 2024. Can large language model agents simulate human trust behavior? In Advances in Neural Information Processing Systems, vo… find
  43. Weixiang Yan, Haitian Liu, Tengxiao Wu, Qian Chen, Wen Wang, Haoyuan Chai, Jiayi Wang, Weishan Zhao, Yixin Zhang, Renjun Zhang, et al. 2024. Clinicallab: Aligning agents for multi-departmental clinical diagnostics in the real world. arXiv preprint arXiv:2406.13890. — [arXiv:2406.13890](https://arxiv.org/abs/2406.13890) find
  44. Computational Linguistics: ACL 2024, pages 13947– 13966, Bangkok, Thailand. Association for Computational Linguistics. find
  45. Jing Zhang, Xiaokang Zhang, Daniel Zhang-Li, Jifan Yu, Zijun Yao, Zeyao Ma, Yiqi Xu, Haohua Wang, Xiaohan Zhang, Nianyi Lin, Sunrui Lu, Juanzi Li, and Jie Tang. 2023. Glm-dialog: Noise-tolerant pre-training for knowledge-grounded dialogue generation. In Proceedings of the 29th ACM SIGKDD Conference… find
  46. Zheyuan Zhang, Daniel Zhang-Li, Jifan Yu, Linlu Gong, Jinchang Zhou, Zhiyuan Liu, Lei Hou, and Juanzi Li. 2024b. Simulating classroom education with llm-empowered agents. arXiv preprint arXiv:2406.19226. find
  47. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing… find
  48. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. 2024. LlamaFactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 400–410, Bangkok, Thail… find our card
  49. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… find
  50. Chenhao Zhang, Renhao Li, Minghuan Tan, Min Yang, Jingwei Zhu, Di Yang, Jiahao Zhao, Guancheng Ye, Chengming Li, and Xiping Hu. 2024a. CPsyCoun: A report-based multi-turn dialogue reconstruction and evaluation framework for Chinese psychological counseling. In Findings of the Association for find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/qiu-2026-interactive-agents-simulating-counselor-client/