Elamur Research⌕
☰
home / papers / 2026 / Causal-ESC: Reliable Policy Learning for Emotional Support Conversation via Causal Inference

Causal-ESC: Reliable Policy Learning for Emotional Support Conversation via Causal Inference

Paper · 2026 · Xv Wang, Wang Zhenyu, Guanyu Zheng et al.

Summary

Causal-ESC targets Emotional Support Conversation (ESC), where a system must choose a counseling strategy and reply in a way that eases a user's distress. The authors argue that recent LLM-based approaches, such as CoT prompting and multi-agent setups, add architectural complexity without explaining why a given strategy helps. They also note that offline training on logged dialogues suffers from bias and from never observing the outcomes of strategies that weren't taken.

The framework splits generation into two stages:

  • Policy stage. A strategy is selected by a policy trained offline with Doubly Robust (DR) learning. DR combines an outcome model with an inverse-propensity-weighted (IPW) estimator. The policy reads structured dialogue features and is rewarded by the user's sentiment gain.
  • Generation stage. An LLM rewrites the reply in the style of the chosen strategy.

Experiments use ESConv and ED. Statistical checks, including Outcome R² and propensity-score analysis, support the claimed causal links. Automatic and human evaluations show gains in empathy and helpfulness over traditional and LLM-based baselines. The work matters because it offers a more interpretable and robust route to strategy selection in sensitive mental-health settings.

Key points

  • Policy inputs. The policy combines dialogue embeddings with explicit, one-hot-encoded features: background (e.g., job crisis), turn number, initial emotional state, current sentiment value, and previous strategy. The reward is the change in emotion (Δe).
  • DR vs. its components (Table 6). DR reaches a policy value of 0.814 with Std 1.47 and strategy ACC of 53.53. DM scores 0.765, Std 1.87, ACC 49.78. IPW scores 0.809 but with a much higher Std of 4.75 and ACC of 51.03. IPW also has a markedly lower Effective Sample Size (ESS).
  • Human evaluation. Causal-ESC beats Sibyl by 7.7% and GPT-4o by 9% overall, though with many ties. Its advantage is clearer against GLHG, KEMP, and MISC. Inter-annotator agreement is moderate (Kappa 0.43), and the result is significant at p = 0.031.
  • Ablations with GPT-4o and DeepSeek backbones. Removing the Empathetic Policy degrades all metrics. Removing Stylized Rewriting mainly lowers diversity (Dist-1/2). DR also outperforms several classic RL baselines.
  • Predicted strategy distribution. The DR policy's predictions track the ground-truth distribution more closely than a history-only baseline, which overuses "Others." Causal-ESC shifts some probability from "Reflection of feelings" toward "Self-disclosure."

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓South China University of TechnologyGuangzhou · China27 works · 1,554 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 works

🧭 Topics 1Research subjects and languages the work is about.Sentiment & emotion · 1,035 works

Cite

Cite as Xv Wang, Wang Zhenyu, Guanyu Zheng and Rui Zhang. 2026. Causal-ESC: Reliable Policy Learning for Emotional Support Conversation via Causal Inference. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.886

@inproceedings{wang2026causal,
  title = {{Causal-ESC: Reliable Policy Learning for Emotional Support Conversation via Causal Inference}},
  author = {Xv Wang, Wang Zhenyu, Guanyu Zheng and Rui Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.886},
  url = {https://doi.org/10.18653/v1/2026.acl-long.886}
}

References

49 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Satanjeev Banerjee and Alon Lavie. 2004. Meteor: an automatic metric for mt evaluation with high levels of correlation with human judgments. Proceedings of ACL-WMT, pages 65–72. find
  2. Heejung Bang and James M Robins. 2005. Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4):962–973. find
  3. Mingxiu Cai, Daling Wang, Shi Feng, and Yifei Zhang. 2024. Empcrl: Controllable empathetic response generation via in-context commonsense reasoning and reinforcement learning. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (L… find
  4. Huiying Cao, Yiqun Zhang, Shi Feng, Xiaocui Yang, Daling Wang, and Yifei Zhang. 2025. Tool-ed: Enhancing empathetic response generation with the tool calling capability of llm. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5305–5320. find
  5. Xinhao Chen, Chong Yang, Man Lan, Li Cai, Yang Chen, Tu Hu, Xinlin Zhuang, and Aimin Zhou. 2024. Cause-aware empathetic response generation via chain-of-thought fine-tuning. arXiv preprint arXiv:2408.11599. — [arXiv:2408.11599](https://arxiv.org/abs/2408.11599) find
  6. Yi Cheng, Wenge Liu, Wenjie Li, Jiashuo Wang, Ruihui Zhao, Bang Liu, Xiaodan Liang, and Yefeng Zheng. 2022. Improving multi-turn emotional support dialogue generation with lookahead strategy planning. arXiv preprint arXiv:2210.04242. find
  7. Douglas Cohen and Janet Strayer. 1996. Empathy in conduct-disordered and comparison youth. Developmental psychology, 32(6):988. find
  8. Jean Decety and Philip L Jackson. 2004. The functional architecture of human empathy. Behavioral and cognitive neuroscience reviews, 3(2):71–100. find
  9. Yang Deng, Lizi Liao, Liang Chen, Hongru Wang, Wenqiang Lei, and Tat-Seng Chua. 2023. Prompting and evaluating large language models for proactive dialogues: Clarification, target-guided, and non-collaboration. arXiv preprint arXiv:2305.13626.
  10. Miroslav Dudík, John Langford, and Lihong Li. 2011. Doubly robust policy evaluation and learning. arXiv preprint arXiv:1103.4601. — [arXiv:1103.4601](https://arxiv.org/abs/1103.4601) find
  11. Linton C Freeman. 1966. Kish: Survey sampling (book review). Social Forces, 45(1):132. find
  12. Scott Fujimoto, David Meger, and Doina Precup. 2019. Off-policy deep reinforcement learning without exploration. In International conference on machine learning, pages 2052–2062. PMLR. find
  13. James J Gross. 1998. The emerging field of emotion regulation: An integrative review. Review of general psychology, 2(3):271–299. find
  14. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  15. Jochen Hartmann. 2022. Emotion english distilroberta-base. See https://huggingface.co/j-hartmann/emotion-english-distilroberta-base. ↗
  16. Yining Hua, Fenglin Liu, Kailai Yang, Zehan Li, Hongbin Na, Yi-han Sheu, Peilin Zhou, Lauren V Moran, Sophia Ananiadou, David A Clifton, and 1 others. 2025. Large language models in mental health care: a scoping review. Current Treatment Options in Psychiatry, 12(1):27. find
  17. Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. 2020. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643. find
  18. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th annual meeting of the…
  19. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and William B Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies…
  20. Qintong Li, Piji Li, Zhaochun Ren, Pengjie Ren, and Zhumin Chen. 2022. Knowledge bridging for empathetic dialogue generation. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 10993–11001. find
  21. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  22. Chia-Wei Liu, Ryan Lowe, Iulian Vlad Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 conference on empirical methods in natu…
  23. June M Liu, Donghao Li, He Cao, Tianhe Ren, Zeyi Liao, and Jiamin Wu. 2023. Chatcounselor: A large language models for mental health support. arXiv preprint arXiv:2309.15461. — [arXiv:2309.15461](https://arxiv.org/abs/2309.15461) find
  24. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3469–3483.
  25. Navonil Majumder, Deepanway Ghosal, Devamanyu Hazarika, Alexander Gelbukh, Rada Mihalcea, and Soujanya Poria. 2022. Exemplars-guided empathetic response generation controlled by the elements of human communication. IEEE Access, 10:77176– 77190. find
  26. Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. Mime: Mimicking emotions for empathetic response generation. arXiv preprint arXiv:2010.01454.
  27. R OpenAI. 2023. Gpt-4 technical report. arxiv 2303.08774. View in Article, 2(5):1.
  28. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  29. Wei Peng, Yue Hu, Luxi Xing, Yuqiang Xie, Yajing Sun, and Yunpeng Li. 2022. Control globally, understand locally: A global-to-local hierarchical graph network for emotional support conversation. In Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI). find
  30. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 5370–5381. find
  31. Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and 1 others. 2021. Recipes for building an open-domain chatbot. In Proceedings of the 16th conference of the european chapter of the association for computational ling…
  32. Ashish Sharma, Adam Miner, David Atkins, and Tim Althoff. 2020. A computational approach to understanding empathy expressed in text-based mental health support. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5263–5276.
  33. Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, and Tao Yu. 2023. One embedder, any task: Instruction-finetuned text embeddings. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1102–1121.
  34. Adith Swaminathan and Thorsten Joachims. 2015. Batch learning from logged bandit feedback through counterfactual risk minimization. The Journal of Machine Learning Research, 16(1):1731–1755. find
  35. Quan Tu, Yanran Li, Jianwei Cui, Bin Wang, Donghong Ji, and Yansong Zhou. 2022. Misc: A mixed strategy-aware model integrating commonsense for emotional support conversation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pages 308–319. find
  36. Cameron Voloshin, Hoang M Le, Nan Jiang, and Yisong Yue. 2019. Empirical study of off-policy policy evaluation for reinforcement learning. arXiv preprint arXiv:1911.06854. — [arXiv:1911.06854](https://arxiv.org/abs/1911.06854) find
  37. Ana Paiva, Iolanda Leite, and Tiago Ribeiro. 2017. Emotion modeling for social robots. In The Oxford Handbook of Affective Computing, pages 296–308. Oxford University Press. find
  38. Lanrui Wang, Jiangnan Li, Chenxu Yang, Zheng Lin, Hongyin Tang, Huan Liu, Yanan Cao, Jingang Wang, and Weiping Wang. 2025. Sibyl: Empowering empathetic dialogue generation in large language models via sensible and visionary commonsense inference. In Proceedings of the 31st International Conference o…
  39. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  40. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533.
  41. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  42. Yangyang Xu, Jinpeng Hu, Zhuoer Zhao, Zhangling Duan, Xiao Sun, and Xun Yang. 2025. Multiagen-tesc: A llm-based multi-agent collaboration framework for emotional support conversation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 4665–4681. find
  43. Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Shaochen Zhong, Bing Yin, and Xia Hu. 2024. Harnessing the power of llms in practice: A survey on chatgpt and beyond. ACM Transactions on Knowledge Discovery from Data, 18(6):1–32.
  44. Li Yuan, Jin Wang, Liang-Chih Yu, and Xuejie Zhang. 2024. Encoding syntactic information into transformers for aspect-based sentiment triplet extraction. IEEE Transactions on Affective Computing, 15(2):722–735. find
  45. Rui Zhang, Zhenyu Wang, and Dongcheng Mai. 2017. Building emotional conversation systems using multi-task seq2seq learning. In National CCF Conference on Natural Language Processing and Chinese Computing, pages 612–621. Springer. find
  46. Rui Zhang, Zhenyu Wang, Mengdan Zheng, Yangyang Zhao, and Zhenhua Huang. 2021. Emotion-sensitive deep dyna-q learning for task-completion dialogue policy learning. Neurocomputing, 459:122–130. find
  47. Chujie Zheng, Sahand Sabour, Jiaxin Wen, and Minlie Huang. 2022. Augesc: Large-scale data augmentation for emotional support conversation with pre-trained language models. arXiv preprint arXiv:2202.13047. — [arXiv:2202.13047](https://arxiv.org/abs/2202.13047) find
  48. Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional chatting machine: Emotional conversation generation with internal and external memory. In Proceedings of the AAAI conference on artificial intelligence, volume 32. find
  49. Jinfeng Zhou, Zhuang Chen, Bo Wang, and Minlie Huang. 2023. Facilitating multi-turn emotional support conversation with positive emotion elicitation: A reinforcement learning approach. arXiv preprint arXiv:2307.07994. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-causal-esc-reliable-policy-learning/