Elamur Research⌕
☰
home / papers / 2023 / Mixing It Up: Inducing Empathy and Politeness using Multiple Behaviour-aware Generators for Conversational Systems

Mixing It Up: Inducing Empathy and Politeness using Multiple Behaviour-aware Generators for Conversational Systems

Paper · 2023 · Mauajama Firdaus, Priyanshu Priya and Asif Ekbal

Summary

The paper addresses customer-care dialogue. An agent's reply should be helpful, and its form of politeness should also suit the customer's mood. The authors split politeness into five behaviours: appreciation, empathy, apology, greeting and assurance. They define a new task of generating replies guided by one of these behaviours, given the dialogue history and the user's affective state.

Their model is a Transformer encoder-decoder with three parts:

  • Affective tracker: a hierarchical context encoder enriched with emotion and sentiment signals. It also predicts emotion and sentiment classes for the context.
  • Behaviour-aware generators: several separate generators, each producing a behaviour-specific representation.
  • Polite generator: a final generator that combines those representations through a weighted sum and writes the reply.

Experiments use the CYCCD customer-care corpus, extended with crowd-sourced behaviour labels. They show more relevant and context-appropriate replies than earlier politeness-generation systems and ablated variants. This matters because the same complaint can be answered in several polite ways, and choosing the right one affects user satisfaction.

Key points

  • The five polite behaviours modelled are Appreciation, Empathy, Apology, Greeting and Assurance. Each has its own behaviour-aware generator.
  • Emotion features come from the output distribution of DeepMoji, which was pretrained on emoji prediction. Explicit sentiment embeddings are added alongside them.
  • The utterance representation is a sum of five parts:
    • the encoder's EOS hidden state;
    • a positional embedding;
    • a speaker (customer vs. agent) embedding;
    • an emotion embedding;
    • a sentiment embedding.
  • Words use GloVe vectors with sine-cosine positional encodings. A dialogue-level Transformer, which receives a prepended QRY token, then encodes the whole context.
  • Evaluation covers only CYCCD, because no other behaviour-annotated data exists, and no LLMs were tried. The error analysis names two failure types: loss of context information, and choosing the wrong behaviour (e.g., thanking a customer who complained about stale food).

Interesting ideas

  • I1. Instead of treating politeness as one style, the authors break it into named sub-behaviours, so one context can be paired with several valid, distinct replies.
  • I2. The user's emotion and sentiment are treated as features attached to every utterance, not as a single label for the whole dialogue. Each turn's affect can then shape what follows.
  • I3. Parallel specialised generators are blended by learned weights. This keeps each behaviour's contribution separate and inspectable before the final mix.
  • I4. The error taxonomy separates "lost content from context" from "right content, wrong social behaviour". These are two independent axes for judging a reply.
  • I5. A speaker embedding marks whose turn is whose, so the encoder can tell the customer's voice from the agent's.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): when generating a reply from the meaning graph, the speech-act layer could pick an explicit politeness strategy (apologise, assure, greet) as a controllable, explainable slot rather than a fuzzy style.
  • I2 (frames): prag could store per-utterance affect (negative/frustrated, positive/joy) next to the speech act, giving implicature and irony detection a turn-level signal.
  • I3 (back): separate realisers per communicative function could each propose a surface form, and the system would record which one dominated and why.
  • I4 (learn): evaluation of generated answers could score factual coverage and pragmatic appropriateness separately, with deliberately mismatched behaviours used as negative controls.
  • I5 (world): the world of a text could tag each statement with its speaker, so that holdings and states asserted by different participants are kept apart.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AlbertaEdmonton · Canada44 works · 3,236 citations🎓Indian Institute of Technology PatnaPatna · India21 works · 374 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

🧭 Topics 1Research subjects and languages the work is about.Dialogue systems · 624 works

Cite

Cite as Mauajama Firdaus, Priyanshu Priya and Asif Ekbal. 2023. Mixing It Up: Inducing Empathy and Politeness using Multiple Behaviour-aware Generators for Conversational Systems. Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.findings-ijcnlp.30

@inproceedings{firdaus2023mixing,
  title = {{Mixing It Up: Inducing Empathy and Politeness using Multiple Behaviour-aware Generators for Conversational Systems}},
  author = {Mauajama Firdaus, Priyanshu Priya and Asif Ekbal},
  booktitle = {Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings)},
  year = {2023},
  doi = {10.18653/v1/2023.findings-ijcnlp.30},
  url = {https://doi.org/10.18653/v1/2023.findings-ijcnlp.30}
}

References

61 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jaime Cesar Acosta. 2009. Using emotion to gain rapport in a spoken dialog system. find
  2. Russell Beale and Chris Creed. 2009. Affective interaction: How emotional agents affect users. International journal of human-computer studies, 67(9):755– 776. ↗
  3. Timothy W Bickmore, Laura M Pfeifer, and Brian W Jack. 2009. Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 1265– 1274. find
  4. Timothy W Bickmore and Rosalind W Picard. 2005. Establishing and maintaining long-term human-computer relationships. ACM Transactions on Computer-Human Interaction (TOCHI), 12(2):293– 327. ↗
  5. Penelope Brown, Stephen C Levinson, and Stephen C Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press. find
  6. Stanley Chen, Douglas H. Beeferman, and Ronald Rosenfeld. 1998. Evaluation metrics for language models. find
  7. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. arXiv preprint arXiv:1306.6078. ↗
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171–4186.
  9. Isabel Dias, Ricardo Rei, Patrícia Pereira, and Luisa Coheur. 2022. Towards a sentiment-aware conversational agent. In Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents, pages 1–3. find
  10. Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. 2017. Question generation for question answering. In Proceedings of the 2017 conference on empirical methods in natural language processing, pages 866– 874.
  11. Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rah-wan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Conference on Empirical Methods in Natural Language Processing (EMNLP).
  12. Julia Fink. 2012. Anthropomorphism and human likeness in the design of robots and human-robot interaction. In International conference on social robotics, pages 199–208. Springer. find
  13. Mauajama Firdaus, Hardik Chauhan, Asif Ekbal, and Pushpak Bhattacharyya. 2020a. Emosen: Generating sentiment and emotion controlled responses in a multimodal dialogue system. IEEE Transactions on Affective Computing, 13(3):1555–1566. ↗
  14. Mauajama Firdaus, Hardik Chauhan, Asif Ekbal, and Pushpak Bhattacharyya. 2020b. Meisd: A multimodal multi-label emotion, intensity and sentiment dialogue dataset for emotion recognition and sentiment analysis in conversations. In Proceedings of the 28th international conference on computational linguistics, pages 4441–4453. find
  15. Mauajama Firdaus, Hardik Chauhan, Asif Ekbal, and Pushpak Bhattacharyya. 2021a. More the merrier: Towards multi-emotion and intensity controllable response generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12821–12829. find
  16. Mauajama Firdaus, Asif Ekbal, and Pushpak Bhattacharyya. 2020c. Incorporating politeness across languages in customer care responses: Towards building a multi-lingual empathetic dialogue agent. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4172–4182. find
  17. Mauajama Firdaus, Asif Ekbal, and Pushpak Bhattacharyya. 2022a. Polise: Reinforcing politeness using user sentiment for customer care response generation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6165– 6175.
  18. Mauajama Firdaus, Umang Jain, Asif Ekbal, and Pushpak Bhattacharyya. 2021b. Seprg: sentiment aware emotion controlled personalized response generation. In Proceedings of the 14th International Conference on Natural Language Generation, pages 353–363. find
  19. Mauajama Firdaus, Arunav Shandilya, Asif Ekbal, and Pushpak Bhattacharyya. 2022b. Being polite: Modeling politeness variation in a personalized dialog agent. IEEE Transactions on Computational Social Systems. ↗
  20. Mauajama Firdaus, Gopendra Vikram Singh, Asif Ekbal, and Pushpak Bhattacharyya. 2023. Affect-gcn: a multimodal graph convolutional network for multi-emotion with intensity recognition and sentiment analysis in dialogues. Multimedia Tools and Applications, pages 1–22. ↗
  21. Mauajama Firdaus, Nidhi Thakur, and Asif Ekbal. 2022c. Sentiment guided aspect conditioned dialogue generation in a multimodal system. In European Conference on Information Retrieval, pages 199–214. Springer. find
  22. Mauajama Firdaus, Naveen Thangavelu, Asif Ekba, and Pushpak Bhattacharyya. 2020d. Persona aware response generation with emotions. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE. find
  23. Mauajama Firdaus, Naveen Thangavelu, Asif Ekbal, and Pushpak Bhattacharyya. 2022d. I enjoy writing and playing, do you: A personalized and emotion grounded dialogue agent using generative adversarial network. IEEE Transactions on Affective Computing. ↗
  24. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. ↗
  25. Swati Gupta, Marilyn A Walker, and Daniela M Romano. 2007. How rude are you?: Evaluating politeness and affect in interaction. In International Conference on Affective Computing and Intelligent Interaction, pages 203–217. Springer. find
  26. Jonathan Herzig, Guy Feigenblat, Michal Shmueli-Scheuer, David Konopnicki, Anat Rafaeli, Daniel Altman, and David Spivak. 2016. Classifying emotions in customer support dialogues in social media. In SIGDIAL Conference, pages 64–73. find
  27. Sathish Reddy Indurthi, Dinesh Raghu, Mitesh M Khapra, and Sachindra Joshi. 2017. Generating natural language question-answer pairs from a knowledge graph using a rnn based question generation model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 376–385. find
  28. Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N Patel. 2018. Evaluating and informing the design of chatbots. In Proceedings of the 2018 designing interactive systems conference, pages 895– 906. find
  29. Q Vera Liao, Matthew Davis, Werner Geyer, Michael Muller, and N Sadat Shami. 2016. What can you do? studying social-agent orientation and agent proactive interactions with an agent for employees. In Proceedings of the 2016 acm conference on designing interactive systems, pages 264–275. find
  30. Chin-Yew Lin. 2004. Rouge: a package for automatic evaluation of summaries.
  31. Aman Madaan, Amrith Setlur, Tanmay Parekh, Barnabas Poczos, Graham Neubig, Yiming Yang, Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhumoye. 2020. Politeness transfer: A tag and generate approach. arXiv preprint arXiv:2004.14257. ↗
  32. Avinash Madasu, Mauajama Firdaus, and Asif Eqbal. 2022. A unified framework for emotion identification and generation in dialogues. arXiv preprint arXiv:2205.15513. ↗
  33. Abraham Harold Maslow. 1943. A theory of human motivation. Psychological review, 50(4):370. ↗
  34. Mary L McHugh. 2012. Interrater reliability: The kappa statistic. Biochemia medica: Biochemia medica, 22(3):276–282. ↗
  35. Asbjørn Følstad and Marita Skjuve. 2019. Chatbots for customer service: user experience and motivation. In Proceedings of the 1st international conference on conversational user interfaces, pages 1–9. find
  36. Hitesh Golchha, Mauajama Firdaus, Asif Ekbal, and Pushpak Bhattacharyya. 2019. Courteously yours: inducing courteous behavior in customer care responses using reinforced pointer generator network. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 851–860. find
  37. Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. arXiv preprint arXiv:1606.03126. ↗
  38. Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022a. Please be polite: Towards building a politeness adaptive dialogue system for goal-oriented conversations. Neurocomputing, 494:242–254. ↗
  39. Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022b. Predicting politeness variations in goal-oriented conversations. IEEE Transactions on Computational Social Systems. ↗
  40. Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2023a. Genpads: Reinforcing politeness in an end-to-end dialogue system. Plos one, 18(1):e0278323. ↗
  41. Kshitij Mishra, Priyanshu Priya, and Asif Ekbal. 2023b. Help me heal: A reinforced polite and empathetic mental health and legal counseling dialogue system for crime victims. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14408–14416. find
  42. Kshitij Mishra, Priyanshu Priya, and Asif Ekbal. 2023c. Pal to lend a helping hand: Towards building an emotion adaptive polite and empathetic counseling conversational agent. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12254– 12271.
  43. Tong Niu and Mohit Bansal. 2018a. Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics, 6:373– 389. ↗
  44. Tong Niu and Mohit Bansal. 2018b. Polite dialogue generation without parallel data. TACL, 6:373–389. ↗
  45. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pages 311–318. Association for Computational Linguistics.
  46. Sashank J Reddi, Satyen Kale, and Sanjiv Kumar. 2019. On the convergence of adam and beyond. 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. ↗
  47. Azlaan Mustafa Samad, Kshitij Mishra, Mauajama Firdaus, and Asif Ekbal. 2022. Empathetic persuasion: reinforcing empathy and persuasiveness in dialogue systems. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 844–856. find
  48. Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2015. Hierarchical neural network generative models for movie dialogues. arXiv preprint arXiv:1507.04808, 7(8):434– 441. ↗
  49. Xiaoyu Shen, Hui Su, Shuzi Niu, and Vera Demberg. 2018. Improving variational encoder-decoders in dialogue generation. In Proceedings of the AAAI conference on artificial intelligence, volume 32. ↗
  50. Weiyan Shi and Zhou Yu. 2018. Sentiment adaptive end-to-end dialog systems. arXiv preprint arXiv:1804.10731. ↗
  51. Gopendra Vikram Singh, Mauajama Firdaus, Asif Ekbal, and Pushpak Bhattacharyya. 2022. Emoint-trans: A multimodal transformer for identifying emotions and intents in social conversations. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:290–300. ↗
  52. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. ↗
  53. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1532–1543.
  54. Zhiliang Tian, Wei Bi, Xiaopeng Li, and Nevin L Zhang. 2019. Learning to abstract for memory-augmented conversational response generation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 3816–3825. find
  55. Johannes Pittermann, Angela Pittermann, and Wolfgang Minker. 2010. Emotion recognition and adaptation in spoken dialogue systems. International Journal of Speech Technology, 13(1):49–60. ↗
  56. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008.
  57. Priyanshu Priya, Mauajama Firdaus, and Asif Ekbal. 2023a. A multi-task learning framework for politeness and emotion detection in dialogues for mental health counselling and legal aid. Expert Systems with Applications, 224:120025. ↗
  58. Priyanshu Priya, Kshitij Mishra, Palak Totala, and Asif Ekbal. 2023b. Partner: A persuasive mental health and legal counselling dialogue system for women and children crime victims. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 6183–6191. International Joint Conferences on Artificial Intelligence Organization. AI for Good. find
  59. Jiancheng Wang, Jingjing Wang, Changlong Sun, Shoushan Li, Xiaozhong Liu, Luo Si, Min Zhang, and Guodong Zhou. 2020a. Sentiment classification in customer service dialogue with topic-aware multi-task learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9177–9184. find
  60. Yi-Chia Wang, Alexandros Papangelis, Runze Wang, Zhaleh Feizollahi, Gokhan Tur, and Robert Kraut. 2020b. Can you be more social? injecting politeness and positivity into task-oriented conversational agents. arXiv preprint arXiv:2012.14653. ↗
  61. Hainan Zhang, Yanyan Lan, Jiafeng Guo, Jun Xu, and Xueqi Cheng. 2018. Reinforcing coherence for sequence to sequence model in dialogue generation. In IJCAI, pages 4567–4573. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/firdaus-2023-mixing-it-up-inducing-empathy/