Elamur Research⌕
☰
home / papers / 2022 / ProsocialDialog: A Prosocial Backbone for Conversational Agents

ProsocialDialog: A Prosocial Backbone for Conversational Agents

Summary

Open-domain chatbots such as GPT-3, BlenderBot and OPT tend to go along with harmful or unethical user statements, or to dodge them with generic deflections. This paper aims to teach agents to push back constructively and in line with social norms. The authors built ProsocialDialog, a large English multi-turn dataset. GPT-3 plays a speaker who raises problematic situations, drawn from morality-related resources. Crowdworkers write replies that steer the conversation toward prosocial behaviour, and each reply is anchored in explicit rules-of-thumb (RoTs). The replies follow guidance from social science: ask before confronting, show empathy, and suggest better alternatives. Each turn also gets a safety label with a written rationale. Using this data, the authors trained two models. Canary infers RoTs from the dialogue context and can act as a safety module. Prost is a dialogue agent built on the PushShift Transformer. Prost's replies were judged more socially acceptable than those of strong baselines, both in-domain and out-of-domain. Canary's RoTs also made off-the-shelf language models more prosocial without any fine-tuning. The work moves dialogue safety beyond detection and toward guided, norm-aware responses.

Key points

  • Dataset scale: ProsocialDialog has 58K dialogues, 331K utterances, 160K unique RoTs, and 497K safety labels with free-form rationales.
  • Safety labels: The new three-way schema (Needs Caution, Needs Intervention, Casual) describes what the agent should do next, rather than rating the content itself.
  • Data collection: Problematic situations were seeded from Social Chemistry, ETHICS and SBIC. GPT-3 then produced the risky turns, and Amazon Mechanical Turk workers wrote, proofread and labelled the prosocial replies, paid about $15 per hour.
  • Released models: Canary is a context-to-RoT generator, and Prost is a socially-informed chatbot. Canary also improves other language models in a zero-shot setting.
  • Limitations: The RoTs mostly reflect US norms held by a narrow pool of annotators. Canary sometimes produces irrelevant RoTs, and Prost can still give incoherent or biased replies.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

90 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 84 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.emnlp-main.267

@inproceedings{kim2022prosocialdialog,
  title = {{ProsocialDialog: A Prosocial Backbone for Conversational Agents}},
  author = {Kim, Hyunwoo and Yu, Youngjae and Jiang, Liwei and Lu, Ximing and Khashabi, Daniel and Kim, Gunhee and Choi, Yejin and Sap, Maarten},
  booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.267},
  url = {https://doi.org/10.18653/v1/2022.emnlp-main.267}
}

References

76 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 2022. Emergency. Wex. Accessed April 14, 2022 [Online]. find
  2. Maurianne Adams, Warren J Blumenfeld, Rosie Castañeda, Heather W Hackman, Madeline L Peters, and Ximena Zúñiga. 2000. Readings for diversity and social justice. Psychology Press. find
  3. Ashutosh Baheti, Maarten Sap, Alan Ritter, and Mark Riedl. 2021. Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts. In EMNLP. find
  4. C. Daniel Batson and Adam A. Powell. 2003. Altruism and Prosocial Behavior. In Handbook of Psychology, 5th edition. John Wiley & Sons, Inc. find
  5. Roy F. Baumeister and Brad J. Bushman. 2017. Social Psychology and Human Nature, 4th edition. Cengage Learning. find
  6. Paul Bloom. 2010. How do Morals Change? Nature, 464(7288):490–490. find
  7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  8. Frances S Chen, Julia A Minson, and Zakary L Tormala. 2010. Tell Me More: The Effects of Expressed Interest on Receptiveness during Dialog. Journal of Experimental Social Psychology, 46(5):850–853. find
  9. Herbert H Clark and Susan E Brennan. 1991. Grounding in communication. In Perspectives on socially shared cognition., pages 127–149. American Psychological Association. find
  10. William Collins. 2022. Prosocial. Collins English Dictionary. Accessed March 23, 2022 [Online]. find
  11. Kate Crawford. 2021. Atlas of AI. Yale University Press. find
  12. Leslie A DeChurch and Michelle A Marks. 2001. Maximizing the benefits of task conflict: The role of conflict management. International Journal of Conflict Management. find
  13. Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. GoEmotions: A Dataset of Fine-Grained Emotions. In ACL. find
  14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. find
  15. Emily Dinan, Gavin Abercrombie, A. Stevie Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, and Verena Rieser. 2022. Safetykit: First aid for measuring safety in open-domain conversational systems. In NAACL. find
  16. Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. 2019. Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack. In EMNLP. find our card
  17. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. Wizard of Wikipedia: Knowledge-Powered Conversational Agents. In ICLR. find
  18. Sarah E Finch and Jinho D Choi. 2020. Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocols. In SIG-Dial. find
  19. Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. 2020. Social Chemistry 101: Learning to Reason about Social and Moral Norms. In EMNLP. find
  20. Sam Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. 2020. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. In Findings of EMNLP. find our card
  21. Karthik Gopalakrishnan, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, and Dilek Hakkani-Tür. 2019. Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations. In Interspeech. find
  22. Jonathan Haidt, Silvia Helena Koller, and Maria G Dias. 1993. Affect, culture, and morality, or is it wrong to eat your dog? Journal of personality and social psychology, 65(4):613. find
  23. Dominik Hangartner, Gloria Gennaro, Sary Alasiri, Nicholas Bahrich, Alexandra Bornhoft, Joseph Boucher, Buket Buse Demirci, Laurenz Derksen, Aldo Hall, Matthias Jochum, et al. 2021. Empathy-based Counterspeech can Reduce Racist Hate Speech in a Social Media Field Experiment. Proceedings of the Natio… find
  24. John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research, 77(1):81– 112. find
  25. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. Aligning AI With Shared Human Values. In ICLR. find our card
  26. Joseph Hoover, Mohammad Atari, Aida Mostafazadeh Davani, Brendan Kennedy, Gwenyth Portillo-Wightman, Leigh Yeh, Drew Kogon, and Morteza Dehghani. 2019. Bound in hatred: The role of group-based morality in acts of hate. find
  27. Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, and Aaron Courville. 2021. Understanding by Understanding Not: Modeling Negation in Language Models. In NAACL. find
  28. Karen Huang, Michael Yeomans, Alison Wood Brooks, Julia Minson, and Francesca Gino. 2017. It doesn’t Hurt to Ask: Question-asking Increases Liking. Journal of personality and social psychology, 113(3):430. find
  29. Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Le Bras Ronan, Maxwell Forbes, Jon Borchardt, Jenny Liang, Oren Etzioni, Maarten Sap, and Yejin Choi. 2021. Delphi: Towards Machine Ethics and Norms. arXiv preprint arXiv:2110.07574. find
  30. Diederik P Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980. find
  31. Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2021. Internet-augmented Dialogue Generation. arXiv preprint arXiv:2107.07566. find our card
  32. Klaus Krippendorff. 2011. Computing Krippendorff’s Alpha-reliability. find
  33. Mucahid Kutlu, Tyler McDonnell, Tamer Elsayed, and Matthew Lease. 2020. Annotator rationales for labeling tasks in crowdsourcing. The journal of artificial intelligence research, 69:143–189. find
  34. Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. In IJC-NLP. find our card
  35. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. Towards Emotional Support Dialog Systems. In ACL. find our card
  36. Andrea Madotto, Zhaojiang Lin, Genta Indra Winata, and Pascale Fung. 2021. Few-Shot Bot: Prompt-Based Learning for Dialogue Systems. arXiv preprint arXiv:2110.08118. — [arXiv:2110.08118](https://arxiv.org/abs/2110.08118) find
  37. Shikib Mehri, Jinho Choi, Luis Fernando D’Haro, Jan Deriu, Maxine Eskenazi, Milica Gasic, Kallirroi Georgila, Dilek Hakkani-Tur, Zekang Li, Verena Rieser, Samira Shaikh, David Traum, Yi-Ting Yeh, Zhou Yu, Yizhe Zhang, and Chen Zhang. 2022. Report from the NSF future directions workshop on automatic… find
  38. A. H. Miller, W. Feng, A. Fisch, J. Lu, D. Batra, A. Bordes, D. Parikh, and J. Weston. 2017. ParlAI: A Dialog Research Software Platform. arXiv:1705.06476. find our card
  39. Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M Khapra. 2018. Towards Exploiting Background Knowledge for Building Conversation Systems. In EMNLP. find our card
  40. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. 2020. Social Bias Frames: Reasoning about Social and Power Implications of Language. In ACL. find our card
  41. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories. In NAACL. find
  42. Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection. In NAACL. find our card
  43. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training Language Models to Follow Instructions with Human Feedback. arXiv preprint arXiv:2203.02155. find
  44. Hilary Silver. 1994. Social exclusion and social solidarity: Three paradigms. Int’l Lab. Rev., 133:531. find
  45. James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. 2015. The Development and Psychometric Properties of LIWC2015. Technical report. find
  46. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. Red Teaming Language Models with Language Models. arXiv preprint arXiv:2202.03286. find our card
  47. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language Models are Unsupervised Multitask Learners. OpenAI blog, 1(8):9. find
  48. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research, 21:1–67. find
  49. M Afzalur Rahim. 2002. Toward a theory of managing organizational conflict. International journal of conflict management. find
  50. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In ACL. find
  51. Rob Reich, Mehran Sahami, and Jeremy M Weinstein. 2021. System error: Where big tech went wrong and how we can reboot. Hodder & Stoughton. find
  52. Sarah T Roberts. 2017. Social media’s silent filter. The Atlantic. find
  53. Carl R. Rogers. 1946. Significant Aspects of Client-centered Therapy. American Psychologist, 1(10):415. find
  54. Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, et al. 2021. Recipes for Building an Open-Domain Chatbot. In EACL. find our card
  55. Eric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston, and Y-Lan Boureau. 2020. Can You Put it All Together: Evaluating Conversational Agents’ Ability to Blend Skills. In ACL. find our card
  56. Miriah Steiger, Timir J Bharucha, Sukrit Venkatagiri, Martin J Riedl, and Matthew Lease. 2021. The psychological Well-Being of content moderators: The emotional labor of commercial moderation and avenues for improving support. In CHI. find
  57. Chloe Rose Stuart-Ulin. 2018. Microsoft’s politically correct chatbot is even worse than its racist one. https://qz.com/1340990/microsofts-politically-correct-chat-bot-is-even-worse-than-its-racist-one/ . Accessed: 2022-4-28. ↗
  58. Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, and Minlie Huang. 2022. On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark. In Findings of ACL. find
  59. Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. 2021. A Word on Machine Ethics: A Response to Jiang et al.(2021). arXiv preprint arXiv:2111.04158. — [arXiv:2111.04158](https://arxiv.org/abs/2111.04158) find
  60. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. LaMDA: Language Models for Dialog Applications. arXiv preprint arXiv:2201.08239. find our card
  61. Jean M. Twenge, Roy F. Baumeister, C. Nathan De-Wall, Natalie J. Ciarocco, and J. Michael Bartels. 2007. Social Exclusion Decreases Prosocial Behavior. Journal of Personality and Social Psychology, 92(1):56. find
  62. Megan Ung, Jing Xu, and Y-Lan Boureau. 2021. Saferdialogues: Taking feedback gracefully after conversational safety failures. arXiv preprint arXiv:2110.07518. find
  63. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of Thought Prompting Elicits Reasoning in Large Language Models. arXiv preprint arXiv:2201.11903. find our card
  64. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and Social Risks of Harm from Language Models. arXiv preprint arXiv:2112.04359. find our card
  65. Anuradha Welivita and Pearl Pu. 2020. A Taxonomy of Empathetic Response Intents in Human Social Conversations. In COLING. find
  66. Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2020. Recipes for Safety in Open-domain Chatbots. arXiv preprint arXiv:2010.07079. find
  67. Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2021. Bot-Adversarial Dialogue for Safe Conversational Agents. In NAACL. find
  68. Michael Yeomans, Julia Minson, Hanne Collins, Frances Chen, and Francesca Gino. 2020. Conversational Receptiveness: Improving Engagement with Opposing Views. Organizational Behavior and Human Decision Processes, 160:131–148. find
  69. Iris Marion Young. 2014. Five faces of oppression. Rethinking power, pages 174–195. find
  70. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing Dialogue Agents: I Have a Dog, Do You Have Pets Too? In ACL. find our card
  71. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. OPT: Open Pre-trained Transformer Language Models. arXiv preprint arXiv:2205.01068. find
  72. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. DialoGPT : Large-Scale Generative Pre-training for Conversational Response Generation. In ACL: System Demonstrations. find our card
  73. Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The design and implementation of xiaoice, an empathetic social chatbot. Computational Linguistics, 46(1):53–93. find
  74. Pei Zhou, Pegah Jandaghi, Hyundong Cho, Bill Yuchen Lin, Jay Pujara, and Xiang Ren. 2021a. Probing Commonsense Explanation in Dialogue Response Generation. In Findings of EMNLP. find
  75. Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi, and Noah A Smith. 2021b. Challenges in Automated Debiasing for Toxic Language Detection. In EACL. find our card
  76. Caleb Ziems, Jane A Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. 2022. The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems. In ACL. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kim-2022-prosocialdialog-prosocial-backbone-conversational-agents/