Elamur Research⌕
☰
home / papers / 2026 / Multi-Session Client-Centered Treatment Outcome Evaluation in Psychotherapy

Multi-Session Client-Centered Treatment Outcome Evaluation in Psychotherapy

Summary

Most LLM-based evaluation of psychotherapy has focused on the therapist and looked at one session at a time. This leaves out the client's own experience and how their condition changes over treatment. The authors propose IPAEval, a framework that estimates treatment outcomes from the client's side by turning clinical interview transcripts into psychometric test scores. In the first step, the model links client statements to individual SCL-90 items, recording the evidence, the symptom, whether it is present, and a justification. In the second step, this reasoning is converted into scores for each symptom dimension. A special value marks symptoms the client has not yet discussed, so the model does not guess about information that was never disclosed. Scores from early and late therapy stages are then compared to measure progress, and context from earlier sessions can be added. To test the framework, the authors built TheraPhase from CPsyCoun, with 400 paired initial and completion-stage records. Across nine open and closed LLMs, IPAEval tracked symptom severity and outcomes better than baseline methods. It offers an interpretable, multi-session alternative to approaches that simulate client responses, such as ClientCAST.

Key points

  • Two-stage prompting: an items-aware reasoning step maps client statements to SCL-90 items. An assessment step then scores each symptom dimension from -1 (not discussed) through 0 (absent) and 1 (minimal) to 2 (clear symptoms with dysfunction).
  • Dataset: TheraPhase, derived from CPsyCoun, contains 400 paired initial and completion-stage client records. It is released on Hugging Face as UTSNLPGroup/TheraPhase.
  • Models: nine open- and closed-source LLMs were evaluated, including OpenAI models, Llama3.1 70B/405B, Mistral 8X7B/8X22B and Qwen2 72B.
  • Ablation: removing items-aware reasoning hurts both symptom detection and outcome prediction, which supports the value of item-level rationales.
  • Output reliability: OpenAI models always produced valid structured output. The open-source models often failed to follow the required Pydantic JSON schema.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/3am66kas7b32

@inproceedings{na2026multi,
  title = {{Multi-Session Client-Centered Treatment Outcome Evaluation in Psychotherapy}},
  author = {Na, Hongbin and Shen, Tao and Yu, Shumao and Chen, Ling},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3am66kas7b32},
  url = {https://doi.org/10.63317/3am66kas7b32}
}

References

58 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mihael Arcan, David-Paul Niland, and Fionn Delahunty. 2024. An assessment on comprehending mental health through large language models. find
  2. Henry I Braun, Douglas N Jackson, and David E Wiley. 2001. Socially desirable responding: The evolution of a construct. In The role of constructs in psychological and educational measurement, pages 61–84. Routledge. find
  3. Siyuan Chen, Mengyue Wu, Kenny Q. Zhu, Kunyao Lan, Zhiling Zhang, and Lyuchun Cui. 2023. Llm-empowered chatbots for psychiatrist and patient simulation: Application and evaluation. find
  4. Yu Ying Chiu, Ashish Sharma, Inna Wanyin Lin, and Tim Althoff. 2024. A computational framework for behavioral assessment of llm therapists. find
  5. Leonard R Derogatis, Ronald S Lipman, and Lino Covi. 1973. Scl-90: an outpatient psychiatric rating scale–preliminary report. Psychopharmacol bull, 9(1):13–28. find
  6. Leonard R Derogatis and Nick Melisaratos. 1983. The brief symptom inventory: an introductory report. Psychological medicine, 13(3):595–605. find
  7. Leonard R Derogatis and Rachael Unger. 2010. Symptom checklist-90-revised. The Corsini encyclopedia of psychology, pages 1–2. find
  8. R. Michael Furr. 2020. Psychometrics in Clinical Psychological Research, Cambridge Handbooks in Psychology, page 54–65. Cambridge University Press. find
  9. Isaac R. Galatzer-Levy, Daniel McDuff, Vivek Natarajan, Alan Karthikesalingam, and Matteo Malgaroli. 2023. The capability of large language models to measure psychiatric functioning. find
  10. Simon B Goldberg, Scott A Baldwin, Kritzia Merced, Derek D Caperton, Zac E Imel, David C Atkins, and Torrey Creed. 2020. The structure of competence: Evaluating the factor structure of the cognitive therapy rating scale. Behavior Therapy, 51(1):113–122. find
  11. Jason T Goodson, Amy W Helstrom, Emily J Marino, and Rachel V Smith. 2017. The impact of service-connected disability and therapist experience on outcomes from prolonged exposure therapy with veterans. Psychological Trauma: Theory, Research, Practice, and Policy, 9(6):647. find
  12. Jonathan Gratch, Ron Artstein, Gale Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, David Traum, Skip Rizzo, and Louis-Philippe Morency. 2014. The distress analysis interview corpus of human and computer interviews. In Proceedings of th… find
  13. Gary Groth-Marnat. 2009. Handbook of psychological assessment. John Wiley & Sons. find
  14. Robert L Hatcher and J Arthur Gillaspy. 2006. Development and validation of a revised short version of the working alliance inventory. Psychotherapy research, 16(1):12–25. find
  15. Derek R Hatfield and Benjamin M Ogles. 2004. The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35(5):485. find
  16. Adele M Hayes and Leigh A Andrews. 2020. A complex systems approach to the study of change in psychotherapy. BMC medicine, 18:1–13. find
  17. Yining Hua, Hongbin Na, Zehan Li, Fenglin Liu, Xiao Fang, David Clifton, and John Torous. 2024. Applying and evaluating large language models in mental health care: A scoping review of human-assessed generative tasks. find
  18. Amanda Jensen-Doss, Emily M Becker Haimes, Ashley M Smith, Aaron R Lyon, Cara C Lewis, Cameo F Stanick, and Kristin M Hawley. 2018. Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Administration and Policy in Mental Health and Mental Health Servi… find
  19. Shaoxiong Ji, Tianlin Zhang, Luna Ansari, Jie Fu, Prayag Tiwari, and Erik Cambria. 2022. MentalBERT: Publicly available pretrained language models for mental healthcare. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 7184–7190, Marseille, France. European Langua… find
  20. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… find
  21. Robert G Johns, Michael Barkham, Stephen Kellett, and David Saxon. 2019. A systematic review of therapist effects: A critical narrative update and refinement to review. Clinical Psychology Review, 67:78–93. find
  22. Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024. Prometheus 2: An open source language model specialized in evaluating other language models. find our card
  23. Kurt Kroenke, Tara W Strine, Robert L Spitzer, Janet BW Williams, Joyce T Berry, and Ali H Mokdad. 2009. The phq-8 as a measure of current depression in the general population. Journal of affective disorders, 114(1-3):163–173. find
  24. Suyeon Lee, Sunghwan Kim, Minju Kim, Dongjin Kang, Dongil Yang, Harim Kim, Minseok Kang, Dayi Jung, Min Hee Kim, Seungbeen Lee, Kyoung-Mee Chung, Youngjae Yu, Dongha Lee, and Jinyoung Yeo. 2024. Cactus: Towards psychological counseling conversations using cognitive behavioral theory. find
  25. Anqi Li, Yu Lu, Nirui Song, Shuai Zhang, Lizhi Ma, and Zhenzhong Lan. 2024a. Automatic evaluation for mental health counseling using llms. find
  26. Zhen Li, Xiaohan Xu, Tao Shen, Can Xu, Jia-Chen Gu, Yuxuan Lai, Chongyang Tao, and Shuai Ma. 2024b. Leveraging large language models for nlg evaluation: Advances and challenges. find
  27. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore. Association for Computational Lin… find
  28. Zheheng Luo, Qianqian Xie, and Sophia Ananiadou. 2023. Chatgpt as a factual inconsistency evaluator for text summarization. find our card
  29. Vincent P. Martin and Jean-Luc Rouas. 2024. Why voice biomarkers of psychiatric disorders are not used in clinical practice? deconstructing the myth of the need for objective diagnosis. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Eva… find
  30. Mark E. Maruish and Robert L. Leahy. 2000. The use of psychological testing for treatment planning and outcome assessment. Journal of Cognitive Psychotherapy, 14:205 – 206. find
  31. Liam Mason, Nick Grey, and David Veale. 2016. My therapist is a student? the impact of therapist experience and client severity on cognitive behavioural therapy outcomes for people with anxiety disorders. Behavioural and Cognitive Psychotherapy, 44(2):193–202. find
  32. Gregory J Meyer, Stephen E Finn, Lorraine D Eyde, Gary G Kay, Kevin L Moreland, Robert R Dies, Elena J Eisman, Tom W Kubiszyn, and Geoffrey M Reed. 2001. Psychological testing and psychological assessment: A review of evidence and issues. American psychologist, 56(2):128. find
  33. Ali Montazeri, Amir Mahmood Harirchi, Mohammad Shariati, Gholamreza Garmaroudi, Mehdi Ebadi, and Abolfazl Fateh. 2003. The 12-item general health questionnaire (ghq-12): translation and validation study of the iranian version. Health and quality of life outcomes, 1:1–4. find
  34. Delroy L Paulhus. 2017. Socially desirable responding on self-reports. Encyclopedia of personality and individual differences, 1(5). find
  35. Verónica Pérez-Rosas, Xinyi Wu, Kenneth Resnicow, and Rada Mihalcea. 2019. What makes a good counselor? learning to distinguish between high-quality and low-quality counseling conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 926–935, Fl… find
  36. Bill Robinson. 2009. When therapist variables and the client’s theory of change meet. Psychotherapy in Australia, 15(4):60–65. find
  37. Carl Rogers. 2012. Client centered therapy (new ed). Hachette UK. find
  38. Gony Rosenman, Lior Wolf, and Talma Hendler. 2024. Llm questionnaire completion for automatic psychiatric assessment. find
  39. Sander Schulhoff, Michael Ilie, Nishant Balepur, Konstantine Kahadze, Amanda Liu, Chenglei Si, Yinheng Li, Aayush Gupta, HyoJung Han, Sevien Schulhoff, Pranav Sandeep Dulepet, Saurav Vidyadhara, Dayeon Ki, Sweta Agrawal, Chau Pham, Gerson Kroiz, Feileen Li, Hudson Tao, Ashay Srivastava, Hevander Da… find
  40. Andrea Spoto, Gioia Bottesi, Ezio Sanavio, and Giulio Vidotto. 2013. Theoretical foundations and clinical implications of formal psychological assessment. Psychotherapy and psychosomatics, 82(3):197–199. find
  41. Llama Team. 2024a. The llama 3 herd of models. find
  42. OpenAI Team. 2024b. Gpt-4 technical report. find our card
  43. Bruce E Wampold and Zac E Imel. 2015. The great psychotherapy debate: The evidence for what makes psychotherapy work. Routledge. find
  44. Jiashuo Wang, Yang Xiao, Yanran Li, Changhe Song, Chunpu Xu, Chenhao Tan, and Wenjie Li. 2024a. Towards a client-centered assessment of llm therapists by client simulation. find
  45. Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. 2024b. Large language models are not fair evaluators. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),… find
  46. Frank W Weathers, B Litz, D Herman, J Juska, and T Keane. 1994. Ptsd checklist—civilian version. Journal of Occupational Health Psychology. find
  47. Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang. 2024. Benchmarking complex instruction-following with multiple constraints composition. find
  48. Zixiu Wu, Simone Balloccu, Vivek Kumar, Rim Helaoui, Diego Reforgiato Recupero, and Daniele Riboni. 2023. Creation, analysis and evaluation of annomi, a dataset of expert-annotated counselling dialogues. Future Internet, 15(3). find
  49. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… find
  50. Stav Yosef, Moreah Zisquit, Ben Cohen, Anat Klomek Brunstein, Kfir Bar, and Doron Friedman. 2024. Assessing motivational interviewing sessions with AI-generated patient simulations. find
  51. In Proceedings of the 9th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2024), pages 1–11, St. Julians, Malta. Association for Computational Linguistics. find
  52. Wei Zhai, Hongzhi Qi, Qing Zhao, Jianqiang Li, Ziqi Wang, Han Wang, Bing Xiang Yang, and Guanghui Fu. 2024. Chinese mentalbert: Domain-adaptive pre-training on social media for chinese mental health text analysis. find
  53. Chenhao Zhang, Renhao Li, Minghuan Tan, Min Yang, Jingwei Zhu, Di Yang, Jiahao Zhao, Guancheng Ye, Chengming Li, and Xiping Hu. 2024. Cpsycoun: A report-based multi-turn dialogue reconstruction and evaluation framework for chinese psychological counseling. find
  54. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023a. A survey of… find our card
  55. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023b. A survey of… find our card
  56. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing… find
  57. Language Resource References Verónica Pérez-Rosas, Xinyi Wu, Kenneth Resnicow, and Rada Mihalcea. 2019. High- vs. low-quality counseling conversations corpus. Dataset derived from public video sources. Transcripts from YouTube/Vimeo demonstration videos; availability varies; see paper for details. find
  58. Zixiu Wu, Simone Balloccu, Vivek Kumar, Rim Helaoui, Diego Reforgiato Recupero, and Daniele Riboni. 2023. Annomi: Expert-annotated counselling dialogues. Dataset. Professionally transcribed MI dialogues with expert annotations; openly available. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/na-2026-multi-session-client-centered-treatment/