Elamur Research⌕
☰
home / papers / 2023 / PAL to Lend a Helping Hand: Towards Building an Emotion Adaptive Polite and Empathetic Counseling Conversational Agent

PAL to Lend a Helping Hand: Towards Building an Emotion Adaptive Polite and Empathetic Counseling Conversational Agent

Paper · 2023 · Kshitij Mishra, Priyanshu Priya and Asif Ekbal

Summary

The paper addresses online mental-health counseling, where a dialogue agent should adjust its tone to the client's emotional state instead of answering in one fixed style. The authors build two counseling corpora with emotion, politeness and empathy labels. EPE-enEIH comes from an English translation of the Hindi EmoInHindi dataset, which covers women and child crime victims. EPE-HLCC comes from the High-quality and Low-quality Counseling Conversations dataset of Motivational Interviewing sessions on addiction and behaviour change. Their agent, PAL, starts from a supervised fine-tuned GPT-2-medium. It is then refined with reinforcement learning using a composite reward. Six preference rewards check that politeness and empathy fit the client's emotion, and two generic rewards favour natural and non-repetitive output. In both automatic and human evaluation, PAL scores above LM, ARDM, EIDM and ablated variants. The ablations show that both reward groups are needed. The work suggests that emotion-conditioned style control can be learned through reward shaping with separate preference models.

Key points

  • en-EmoInHindi: 1,814 dialogues, machine-translated with Google Translate and then post-edited by four translators. Fluency/Adequacy rose from 3.23/3.10 to 4.65/4.48 on a 1–5 scale.
  • HLCC has 258 sessions (154 high-quality, 104 low-quality). Both corpora were labelled by three annotators with linguistics backgrounds.
  • Emotion labels were derived from EMPATHETICDIALOGUES. Its 32 emotions were merged into 11 classes, a set called EmpDial-merged, to get a more balanced class distribution.
  • The reward combines six preference rewards and two generic rewards. It is computed by six transformer preference models: two generators and four classifiers.
  • Weak spots:
    • Emotion-politeness and emotion-empathy consistency reached only 73.7% and 75.9%.
    • Training needs a 40 GB GPU.
    • The model drifts when the client gives very short replies.
    • It tends to ask too many questions.

Interesting ideas

  • I1. The translation process lets humans score every machine-translated utterance but post-edit only those scoring 3 or lower, which focuses expensive human effort where quality is worst.
  • I2. Fine-grained labels were merged (32 emotions to 11) to balance classes before training a classifier, trading granularity for reliability.
  • I3. The reward is split into independent, swappable preference modules plus generic quality terms, and each group is ablated to show it contributes.
  • I4. Response style is conditioned on the interlocutor's detected emotion, so the same content gets different politeness and empathy depending on the user's state.
  • I5. The error analysis traced failures on terse inputs ("Yes", "No") to training data dominated by long utterances.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver-data pipelines could route only low-agreement or low-confidence items to the LLM teacher or human tier, rather than re-checking everything.
  • I2 (frames): Speech-act or affect inventories in prag could use a coarse layer for robust prediction, with the fine distinctions kept as an optional refinement.
  • I3 (back): Generation from the meaning graph could be scored by separate explainable checkers (role fidelity, register, non-repetition), each reported and ablated on its own.
  • I4 (back): Realisation could choose register and hedging from a speaker-state attribute in the world model rather than using one fixed style.
  • I5 (learn): Evaluation suites could include deliberately short or degenerate inputs as negative controls to expose distribution gaps in parsers and readers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Indian Institute of Technology PatnaPatna · India21 works · 374 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksDialogue systems · 624 works

Cite

Cite as Kshitij Mishra, Priyanshu Priya and Asif Ekbal. 2023. PAL to Lend a Helping Hand: Towards Building an Emotion Adaptive Polite and Empathetic Counseling Conversational Agent. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-long.685

@inproceedings{mishra2023pal,
  title = {{PAL to Lend a Helping Hand: Towards Building an Emotion Adaptive Polite and Empathetic Counseling Conversational Agent}},
  author = {Kshitij Mishra, Priyanshu Priya and Asif Ekbal},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.685},
  url = {https://doi.org/10.18653/v1/2023.acl-long.685}
}

References

47 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72.
  2. Timothy W Bickmore and Rosalind W Picard. 2004. Towards caring machines. In CHI’04 extended abstracts on Human factors in computing systems, pages 1489–1492. find
  3. Penelope Brown, Stephen C Levinson, and Stephen C Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press. find
  4. Peter F Brown, Stephen A Della Pietra, Vincent J Della Pietra, Jennifer C Lai, and Robert L Mercer. 1992. An estimate of an upper bound for the entropy of english. Computational Linguistics, 18(1):31–40. ↗
  5. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of ACL. ↗
  6. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference on Human Language Technology Research, pages 138–145. find
  7. Susan V Eisen, Deborah J Youngman, Mollie C Grob, and Diana L Dill. 1992. Alcohol, drugs, and psychiatric disorders: A current view of hospitalized adolescents. Journal of Adolescent Research, 7(2):250– 265. ↗
  8. Robert Elliott, Arthur C Bohart, Jeanne C Watson, and David Murphy. 2018. Therapist empathy and client outcome: An updated meta-analysis. Psychotherapy, 55(4):399. ↗
  9. Bo Feng, Siyue Li, and Na Li. 2016. Is a profile worth a thousand words? how online support-seeker’s profile features may influence the quality of received support messages. Communication Research, 43(2):253– 276. find
  10. Asma Ghandeharioun, Daniel McDuff, Mary Czerwinski, and Kael Rowan. 2019. Towards understanding emotional intelligence for behavior change chatbots. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), pages 8–14. IEEE. find
  11. James Gibson, Dogan Can, Bo Xiao, Zac E Imel, David C Atkins, Panayiotis Georgiou, and Shrikanth Narayanan. 2016. A deep learning approach to modeling empathy in addiction counseling. Commitment, 111(2016):21. find
  12. Emily A Holmes, Ata Ghaderi, Catherine J Harmer, Paul G Ramchandani, Pim Cuijpers, Anthony P Morrison, Jonathan P Roiser, Claudi LH Bockting, Rory C O’Connor, Roz Shafran, et al. 2018. The lancet psychiatry commission on psychological treatments research in tomorrow’s science. The Lancet Psychiatry, 5(3):237–286. ↗
  13. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751. ↗
  14. Elizabeth M Huffman, Dimitrios I Athanasiadis, Nicholas E Anton, Lindsay A Haskett, Dominique L Doster, Dimitrios Stefanidis, and Nicole K Lee. 2021. How resilient is your team? exploring healthcare providers’ well-being during the covid-19 pandemic. The American Journal of Surgery, 221(2):277–284. ↗
  15. Paul Jaccard. 1912. The distribution of the flora in the alpine zone. 1. New phytologist, 11(2):37–50. find
  16. Neil Katz and Kevin McNulty. 1994. Reflective listening. Retrieved February, 4:2021. find
  17. Dean G Kilpatrick and Ron Acierno. 2003. Mental health needs of crime victims: Epidemiology and outcomes. Journal of Traumatic Stress: Official Publication of The International Society for Traumatic Stress Studies, 16(2):119–132. ↗
  18. Dean G Kilpatrick, Connie L Best, Lois J Veronen, Angelynne E Amick, Lorenz A Villeponteaux, and Gary A Ruff. 1985. Mental health correlates of criminal victimization: a random community survey. Journal of consulting and clinical psychology, 53(6):866. ↗
  19. Junhan Kim, Yoojung Kim, Byungjoon Kim, Sukyung Yun, Minjoon Kim, and Joongseek Lee. 2018. Can a machine tend to teenagers’ emotional needs? a study with conversational agents. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, pages 1–6. find
  20. Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics, 22(1):79–86. ↗
  21. Fei-Tzin Lee, Derrick Hull, Jacob Levine, Bonnie Ray, and Kathleen McKeown. 2019. Identifying therapist conversational actions across diverse psychotherapeutic approaches. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology, pages 12–23. find
  22. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. ↗
  23. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  24. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  25. Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In International Conference on Learning Representations. ↗
  26. Gale M Lucas, Jonathan Gratch, Aisha King, and Louis-Philippe Morency. 2014. It’s only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior, 37:94–100. ↗
  27. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. ↗
  28. W Mieleszczenko-Kowszewicz, K Warpechowski, K Zieliński, R Nielek, and A Wierzbicki. 2022. Tell me how you feel: Designing emotion-aware voicebots to ease pandemic anxiety in aging citizens. arXiv preprint arXiv:2207.10828. ↗
  29. Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022a. Please be polite: Towards building a politeness adaptive dialogue system for goal-oriented conversations. Neurocomputing, 494:242–254. ↗
  30. Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2023. Genpads: Reinforcing politeness in an end-to-end dialogue system. Plos one, 18(1):e0278323. ↗
  31. Kshitij Mishra, Azlaan Mustafa Samad, Palak Totala, and Asif Ekbal. 2022b. Pepds: A polite and empathetic persuasive dialogue system for charity donation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 424–440.
  32. Robert R Morris, Kareem Kouddous, Rohan Kshirsagar, and Stephen M Schueller. 2018. Towards an artificially empathic conversational agent for mental health applications: system design and user perceptions. Journal of medical Internet research, 20(6):e10148. ↗
  33. Joseph Newbold, Gavin Doherty, Sean Rintel, and Anja Thieme. 2019. Politeness strategies in the design of voice agents for mental health. find
  34. Mario R Paredes, Vanessa Apaolaza, Cristóbal Fernandez-Robin, Patrick Hartmann, and Diego Yañez-Martinez. 2021. The impact of the covid-19 pandemic on subjective mental well-being: The interplay of perceived threat, future anxiety and resilience. Personality and Individual Differences, 170:110455. ↗
  35. Verónica Pérez-Rosas, Xinyi Wu, Kenneth Resnicow, and Rada Mihalcea. 2019. What makes a good counselor? learning to distinguish between high-quality and low-quality counseling conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 926–935. find
  36. Yada Pruksachatkun, Sachin R Pendse, and Amit Sharma. 2019. Moments of change: Analyzing peer-based cognitive support in online mental health forums. In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1–13. find
  37. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. find
  38. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381, Florence, Italy. Association for Computational Linguistics. find
  39. Elliot Robert, Arthur C Bohart, JC Watson, and LS Greenberg. 2011. Empathy. Psychotherapy, 48(1):43–49. find
  40. Stephen Ross and Eric Peselow. 2012. Co-occurring psychotic and addictive disorders: neurobiology and diagnosis. Clinical neuropharmacology, 35(5):235– 243. ↗
  41. Tulika Saha, Vaibhav Gakhreja, Anindya Sundar Das, Souhitya Chakraborty, and Sriparna Saha. 2022a. Towards motivational and empathetic response generation in online mental health support. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2650–2656. find
  42. World Health Organization et al. 2021. Mental health atlas 2020. find
  43. Tulika Saha, Saichethan Reddy, Anindya Das, Sriparna Saha, and Pushpak Bhattacharyya. 2022b. A shoulder to cry on: Towards a motivational virtual assistant for assuaging mental agony. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2436–2449. ↗
  44. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. ↗
  45. Azlaan Mustafa Samad, Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022. Empathetic persuasion: Reinforcing empathy and persuasiveness in dialogue systems. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 844–856. find
  46. John C Norcross. 2002. Psychotherapy relationships that work: Therapist contributions and responsiveness to patients. Oxford University Press. find
  47. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mishra-2023-pal-lend-helping-hand-building/