Elamur Research⌕
☰
home / papers / 2026 / I Understand How You Feel: Enhancing Deeper Emotional Support Through Multilingual Emotional Validation in Dialogue System

I Understand How You Feel: Enhancing Deeper Emotional Support Through Multilingual Emotional Validation in Dialogue System

Summary

Emotional validation means signalling to a user that their feelings are understandable. It helps in therapy, but dialogue research has given it little attention, and earlier systems relied on fixed phrase lists tested only in Japanese. The authors also note that LLMs tend to agree with users too readily, which risks over-validation.

They split the problem into three subtasks: recognising validating replies, deciding when to validate, and generating validating replies. They release two resources:

  • M-EDESConv: an English–Japanese text corpus of about 120k turns, built from translated empathetic-dialogue data.
  • M-TESC: a spoken-dialogue test set.

Labels come from a small manually annotated subset plus an XLM-RoBERTa classifier for the rest. For the timing subtask they propose MEGUMI, which combines frozen multilingual semantic features with emotion features specific to each language.

The EmoValidBench benchmark shows that LLMs produce varied, contextually fitting replies but still understand emotion poorly. In a joint timing-and-generation test, using MEGUMI to decide when an LLM should validate reduced unnecessary validation. This matters for building empathetic agents that validate only when it is warranted.

Key points

  • Corpus construction: Sources are EmpatheticDialogues, ESConv and the spoken TESC. Japanese versions were produced with GPT-4.1-mini, and TESC was translated into English. Translation quality reached COMETKiwi ≈0.847 and a human rating of 5.63/7.
  • Annotation: About 3,000 responses per source were labelled by hand. A fine-tuned xlm-roberta-large classifier labelled the rest (macro-F1 85.28). Crowd workers agreed with 85% of a sample of automatic labels (IAA 0.752).
  • MEGUMI architecture:
    • frozen XLM-RoBERTa-large for semantics;
    • ModernBERT-large tuned on GoEmotions for English emotion;
    • LUKE-Japanese-large tuned on WRIME for Japanese emotion;
    • cross-lingual emotion attention (EEMA) combined through a gated multimodal unit.
  • EmoValidBench: Benchmarks GPT-4.1 Nano and Llama-3.1 8B using semantic, diversity, empathy and LLM-as-judge metrics. Human-written references scored best on most dimensions.
  • Joint evaluation (200 utterances): MEGUMI+LLM achieved the best timing precision (64.20) and message-level macro-F1 (61.15) among automatic systems. An always-validating LLM had the highest recall (92.00) but over-validated. Human judgement remained the upper bound (message-level F1 70.82).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.sigdial-1.16

@inproceedings{pang2026i,
  title = {{I Understand How You Feel: Enhancing Deeper Emotional Support Through Multilingual Emotional Validation in Dialogue System}},
  author = {Pang, Zi Haur and Fu, Yahui and Inoue, Koji and Kawahara, Tatsuya},
  booktitle = {Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue},
  year = {2026},
  doi = {10.18653/v1/2026.sigdial-1.16},
  url = {https://doi.org/10.18653/v1/2026.sigdial-1.16}
}

References

74 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Deeksha Adiani, Kelley Colopietro, Joshua Wade, Miroslava Migovich, Timothy J Vogus, and Nilanjan Sarkar. 2023. Dialogue act classification via transfer learning for automated labeling of interviewee responses in virtual reality job interview training platforms for autistic individuals. Signals, 4(2… find
  2. John Arevalo, Thamar Solorio, Manuel Montes-y Gómez, and Fabio A González. 2017. Gated multimodal units for information fusion. arXiv preprint arXiv:1702.01992. — [arXiv:1702.01992](https://arxiv.org/abs/1702.01992) find
  3. Tadesse Destaw Belay, Ahmed Haj Ahmed, Alvin Grissom II, Iqra Ameer, Grigori Sidorov, Olga Kolesnikova, and Seid Muhie Yimam. 2025. find
  4. CULEMO: Cultural lenses on emotion - benchmarking LLMs for cross-cultural emotion understanding. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18894–18909, Vienna, Austria. Association for Computational Linguistics. Bojana B… find
  5. Naomi Breslau, Ronald C Kessler, Howard D Chilcoat, Lonni R Schultz, Glenn C Davis, and Patricia An-dreski. 1998. Trauma and posttraumatic stress disorder in the community: the 1996 detroit area survey of trauma. Archives of general psychiatry, 55(7):626– 632. find
  6. Mingxiu Cai, Daling Wang, Shi Feng, and Yifei Zhang. 2024. Pecer: Empathetic response generation via dynamic personality extraction and contextual emotional reasoning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10631–10635. IEEE. find
  7. Lea Canales, Carlo Strapparava, Ester Boldrini, and Patricio Martínez-Barco. 2016. Innovative semi-automatic methodology to annotate emotional corpora. In Proceedings of the Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media (PEOPLES), pages 91–100. find
  8. Amanda Carson-Wong, Christopher D Hughes, and Shireen L Rizvi. 2018. The effect of therapist use of validation strategies on change in client emotion in individual dbt treatment sessions. Personality Disorders: Theory, Research, and Treatment, 9(2):165. find
  9. Jacky Casas, Timo Spring, Karl Daher, Elena Mugellini, Omar Abou Khaled, and Philippe Cudré-Mauroux. 2021. Enhancing conversational agents with empathic abilities. In Proceedings of the 21st ACM international conference on intelligent virtual agents, pages 41–47. find
  10. Mao Yan Chen, Siheng Li, and Yujiu Yang. 2022. EmpHi: Generating empathetic responses with human-like intents. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1063–1074, Seattle, United States. A… find
  11. Julie W Childers, Hailey Bulls, and Robert Arnold. 2023. Beyond the nurse acronym: the functions of empathy in serious illness conversations. Journal of pain and symptom management, 65(4):e375–e379. find
  12. Patricia M Clancy, Sandra A Thompson, Ryoko Suzuki, and Hongyin Tao. 1996. The conversational use of reactive tokens in english, japanese, and mandarin. Journal of pragmatics, 26(3):355–387. find
  13. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  14. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2475–2485. find
  15. Hannah Sophie Daniel. 2023. Exploring emotional validation in cross-cultural management: a case study. find
  16. Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. GoEmotions: A dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4040–4054, Online. Association for Computational… find
  17. Sara M Edlund, Maria L Carlsson, Steven J Linton, Alan E Fruzzetti, and Maria Tillfors. 2015. I see you’re in pain–the effects of partner validation on emotions in people with chronic pain. Scandinavian Journal of Pain, 6(1):16–21. find
  18. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3. find
  19. Joshua Conrad Jackson, Joseph Watts, Teague R Henry, Johann-Mattis List, Robert Forkel, Peter J Mucha, Simon J Greenhill, Russell D Gray, and Kristen A Lindquist. 2019. Emotion semantics show both cultural variation and universal structure. Science, 366(6472):1517–1522. find
  20. Deborah Johanson, Ho Seok Ahn, Rishab Goswami, Kazuki Saegusa, and Elizabeth Broadbent. 2023. The effects of healthcare robot empathy statements and head nodding on trust and satisfaction: a video study. ACM Transactions on Human-Robot Interaction, 12(1):1–21. find
  21. Dongjin Kang, Sunghwan Kim, Taeyoon Kwon, Seungjun Moon, Hyunsouk Cho, Youngjae Yu, Dongha Lee, and Jinyoung Yeo. 2024. Can large language models be good emotional supporter? mitigating preference bias on emotional support conversation. In Proceedings of the 62nd Annual Meeting of the Association fo… find
  22. Heejung S Kim, David K Sherman, and Shelley E Taylor. 2008. Culture and social support. American psychologist, 63(6):518. find
  23. Lauren Fonteyn, Enrique Manjavacas, Nina Haket, Aletta G Dorst, and Eva Kruijt. 2024. Could this be next for corpus linguistics? methods of semi-automatic data annotation with contextualized word embeddings. Linguistics Vanguard, 10(1):587–602. find
  24. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213. find our card
  25. Yahui Fu, Chenhui Chu, and Tatsuya Kawahara. 2024. Styemp: Stylizing empathetic response generation via multi-grained prefix encoder and personality reinforcement. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 172–185. find
  26. Janice R Kuo, Skye Fitzpatrick, Jennifer Ip, and Amanda Uliaszek. 2022. The who and what of validation: an experimental examination of validation and invalidation of specific emotions and the moderating effect of emotion dysregulation. Borderline Personality Disorder and Emotion Dysregulation, 9(1):… find
  27. Yahui Fu, Koji Inoue, Chenhui Chu, and Tatsuya Kawahara. 2023. Reasoning before responding: Integrating commonsense-based causality explanation for empathetic response generation. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 645–656. find
  28. John A Lambie and Anja Lindberg. 2016. The role of maternal emotional validation and invalidation on children’s emotional awareness. Merrill-Palmer Quarterly, 62(2):129–157. find
  29. Jun Gao, Yuhan Liu, Haolin Deng, Wei Wang, Yu Cao, Jiachen Du, and Ruifeng Xu. 2021. Improving empathetic response generation by recognizing emotion cause in conversations. In Findings of the association for computational linguistics: EMNLP 2021, pages 807–819. find
  30. Jiseung Hong, Grace Byun, Seungone Kim, and Kai Shu. 2025. Measuring sycophancy of language models in multi-turn dialogues. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 2239–2259, Suzhou, China. Association for Computational Linguistics. find
  31. Young-Jun Lee, Chae-Gyun Lim, and Ho-Jin Choi. 2022. Does gpt-3 generate empathetic dialogues? a novel in-context example selection method and automatic evaluation metric for empathetic dialogue generation. In 29th COLING, pages 669–683. find
  32. Iolanda Leite, André Pereira, Samuel Mascarenhas, Carlos Martinho, Rui Prada, and Ana Paiva. 2013. The influence of empathy in human–robot relations. International journal of human-computer studies, 71(3):250–260. find
  33. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pag… find our card
  34. Qintong Li, Piji Li, Zhaochun Ren, Pengjie Ren, and Zhumin Chen. 2022. Knowledge bridging for empathetic dialogue generation. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 10993–11001. find
  35. Zhaojiang Lin, Andrea Madotto, Jamin Shin, Peng Xu, and Pascale Fung. 2019. MoEL: Mixture of empathetic listeners. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages… find our card
  36. Marsha M Linehan. 1997. Validation and psychotherapy. American Psychological Association. find
  37. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan… find our card
  38. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. find
  39. Navonil Majumder, Pengfei Hong, Shanshan Peng, Jiankun Lu, Deepanway Ghosal, Alexander Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. MIME: MIMicking emotions for empathetic response generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pag… find our card
  40. Audrey Marcoux, Marie-Hélène Tessier, and Philip L Jackson. 2024. Nonverbal behaviors perceived as most empathic in a simulated medical context. Computers in Human Behavior, 157:108268. find
  41. Robert R Morris, Kareem Kouddous, Rohan Kshirsagar, and Stephen M Schueller. 2018. Towards an artificially empathic conversational agent for mental health applications: system design and user perceptions. Journal of medical Internet research, 20(6):e10148. find
  42. Hikaru Oishi, Mika Enomoto, Keiko Ochi, and Yasunari Obuchi. 2021. Design and basic analysis of the tut emotional storytelling corpus. In 2021 24th Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-CO… find
  43. Zi Haur Pang, Yahui Fu, Yuan Gao, and Tatsuya Kawahara. 2026. Paralinguistic emotion-aware validation timing detection in japanese empathetic spoken dialogue. In ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). find
  44. Zi Haur Pang, Yahui Fu, Divesh Lala, Keiko Ochi, Koji Inoue, and Tatsuya Kawahara. 2023. Prediction of validating response from emotional storytelling corpus. In The 37th Annual Conference of the Japanese Society for Artificial Intelligence, pages 2O5OS2a03– 2O5OS2a03. find
  45. Zi Haur Pang, Yahui Fu, Divesh Lala, Keiko Ochi, Koji Inoue, and Tatsuya Kawahara. 2024. Acknowledgment of emotional states: Generating validating responses for empathetic dialogue. 14th International Workshop on Spoken Dialogue System Technology. find
  46. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. find our card
  47. Robert Plutchik. 2001. The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American scientist, 89(4):344–350. find
  48. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381, Florence, Italy. Association for Com… find
  49. Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and André F. T. Martins. 2022. find
  50. CometKiwi: IST-unbabel 2022 submission for the quality estimation shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 634–645, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find our card
  51. Stephen Rollnick and William R Miller. 1995. What is motivational interviewing? Behavioural and cognitive Psychotherapy, 23(4):325–334. find
  52. Shanequa S Roscoe-Nelson and Geoffrey A Silvera. 2024. Is timing everything?: The role of time on the relationship between patient-centered communication and provider empathy. Patient Experience Journal, 11(2):36–43. find
  53. Andre Rusli and Makoto Shishido. 2024. An experimental evaluation of japanese tokenizers for sentiment-based text classification. The 27th Annual Meeting of the Association for Natural Language Processing. find
  54. Sahand Sabour, Chujie Zheng, and Minlie Huang. 2022. Cem: Commonsense-aware empathetic response generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11229– 11237. find
  55. Kendra S Wasson Simpson, Anna Gallagher, Scott T Ronis, David AA Miller, and Kate C Tilleczek. 2022. Youths’ perceived impact of invalidation and validation on their mental health treatment journeys. Administration and Policy in Mental Health and Mental Health Services Research, pages 1–14. find
  56. Aadesh Salecha, Molly E Ireland, Shashanka Subrahmanya, João Sedoc, Lyle H Ungar, and Johannes C Eichstaedt. 2024. Large language models show human-like social desirability biases in survey responses. arXiv preprint arXiv:2405.06058. — [arXiv:2405.06058](https://arxiv.org/abs/2405.06058) find
  57. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. 2023. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308.03958. find
  58. Azlaan Mustafa Samad, Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022. Empathetic persuasion: reinforcing empathy and persuasiveness in dialogue systems. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 844–856. find
  59. Anuradha Welivita and Pearl Pu. 2020. A taxonomy of empathetic response intents in human social conversations. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4886– 4899, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  60. Ashish Sharma, Adam Miner, David Atkins, and Tim Althoff. 2020. A computational approach to understanding empathy expressed in text-based mental health support. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5263–5276. find our card
  61. Anuradha Welivita, Chun-Hung Yeh, and Pearl Pu. 2023. Empathetic response generation for distress support. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 632–644. find
  62. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, Kravec Shauna, Maxwell Timothy, McCandlish Sam, Ndousse Kamal, Rausch Oliver, Schiefer Nicholas, Yan Da, Zhang Miranda, and Perez Ethan. 2023. Tow… find our card
  63. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373. find our card
  64. Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, and 13 others. 2025. MMLU-ProX: A multilingual benchmark for advanced large l… find
  65. Hui Yang, Alistair Willis, Anne De Roeck, and Bashar Nuseibeh. 2012. A hybrid model for automatic emotion recognition in suicide notes. Biomedical informatics insights, 5:BII–S8948. find
  66. Hiroaki Sugiyama, Masahiro Mizukami, Tsunehiro Arimoto, Hiromi Narimatsu, Yuya Chiba, Hideharu Nakajima, and Toyomi Meguro. 2023. Empirical analysis of training strategies of transformer-based japanese chit-chat systems. In 2022 ieee spoken language technology workshop (slt), pages 685–691. IEEE. find
  67. Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. Paws-x: A cross-lingual adversarial dataset for paraphrase identification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processin… find our card
  68. Yoichi Takenaka. 2025. Performance evaluation of emotion classification in japanese using roberta and deberta. arXiv preprint arXiv:2505.00013. — [arXiv:2505.00013](https://arxiv.org/abs/2505.00013) find
  69. Tenggan Zhang, Xinjie Zhang, Jinming Zhao, Li Zhou, and Qin Jin. 2024. ESCoT: Towards interpretable emotional support dialogue systems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13395–13412, Bangkok, Thailand. Associatio… find
  70. Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the conference. Association for computational linguistics. Meeting, volume 2019, page 6558. find our card
  71. Qing Wang, Shuyuan Peng, Zhiyuan Zha, Xue Han, Chao Deng, Lun Hu, and Pengwei Hu. 2023. Enhancing the conversational agent with an emotional support system for mental health digital therapeutics. Frontiers in Psychiatry, 14:1148534. find
  72. Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. find
  73. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Zhang Hao, E. Gonzalez Joseph, and Stoica Ion. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–4… find
  74. Peixiang Zhong, Chen Zhang, Hao Wang, Yong Liu, and Chunyan Miao. 2020. Towards persona-based empathetic conversational models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6556–6566, Online. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pang-2026-i-understand-how-you-feel/