Elamur Research⌕
☰
home / papers / 2021 / Can Machines Learn Morality? The Delphi Experiment

Can Machines Learn Morality? The Delphi Experiment

Paper · 2021 · Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula et al.

Summary

AI systems increasingly make choices with ethical consequences, yet large language models trained on web text have no explicit grasp of everyday moral norms. The authors present Delphi, a model that predicts how people would typically judge ordinary situations. It is framed as descriptive ethics and loosely grounded in Rawls's bottom-up "decision procedure" and reflective equilibrium. Delphi starts from UNICORN, a T5-11B model tuned for commonsense QA. It is then trained on COMMONSENSE NORM BANK, which merges five existing crowdsourced resources into one format. Given a situation, the model returns either a yes/no verdict or a short free-text judgment. On held-out data it clearly beats GPT-3 and remains sensitive to small changes in context. The paper is equally frank about weaknesses: Delphi inherits social biases and makes inconsistent calls, and its annotators were mostly white and US-based. The authors still show practical value, using Delphi as a component in other systems. The work matters as an early, openly tested resource for computational moral reasoning that also maps its own failure modes.

Key points

  • COMMONSENSE NORM BANK holds about 1.7M crowdsourced ethical judgments drawn from SOCIAL CHEM, SOCIAL BIAS FRAMES, MORAL STORIES, ETHICS and SCRUPLES.
  • On unseen NORM BANK examples, Delphi is correct 92.8% of the time, compared with 60.2% for GPT-3.
  • The public demo (delphi.allenai.org, now running the improved Delphi+) has received more than four million queries.
  • Downstream uses include:
    • a few-shot hate speech detector,
    • morally guided story generation,
    • transfer to other moral frameworks.
  • Bias was probed with Universal Declaration of Human Rights (UDHR) items across 213 identities grouped into 12 categories. The authors added identity-related training data to make Delphi+ more robust against these biases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Cite

Cite as Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini and Yejin Choi. 2021. Can Machines Learn Morality? The Delphi Experiment. arXiv:2110.07574.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2110.07574 · arXiv 2110.07574

@misc{jiang2021can,
  title = {{Can Machines Learn Morality? The Delphi Experiment}},
  author = {Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan {Le Bras}, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini and Yejin Choi},
  year = {2021},
  doi = {10.48550/arxiv.2110.07574},
  eprint = {2110.07574},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2110.07574}
}

References

128 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Saleema Amershi, Maya Cakmak, W. Knox, and Todd Kulesza. Power to the people: The role of humans in interactive machine learning. AI Magazine, 35:105–120, 12 2014. doi: 10.1609/aima g.v35i4.2513. — [doi:10.1609/aima](https://doi.org/10.1609/aima) find
  2. Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. Guidelines for human-ai interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Comput… — [doi:10.1145/3290605.3300233](https://doi.org/10.1145/3290605.3300233) find
  3. Prithviraj Ammanabrolu, Liwei Jiang, Maarten Sap, Hanna Hajishirzi, and Yejin Choi. Aligning to social norms and values in interactive narratives. In NAACL, 2022. find
  4. Susan Leigh Anderson. Asimov’s “three laws of robotics” and machine metaethics. Ai & Society, 22(4):477–493, 2008. find
  5. Alex Andonian, Quentin Anthony, Stella Biderman, Sid Black, Preetham Gali, Leo Gao, Eric Hallahan, Josh Levy-Kramer, Connor Leahy, Lucas Nestler, Kip Parker, Michael Pieler, Shivanshu Purohit, Tri Songz, Phil Wang, and Samuel Weinbach. GPT-NeoX: Large scale autoregressive language modeling in pytorc… find
  6. Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-François Bonnefon, and Iyad Rahwan. The Moral Machine experiment. Nature, 2018. find
  7. Edmond Awad, Sydney Levine, Michael Anderson, Susan Leigh Anderson, Vincent Conitzer, M.J. Crockett, Jim A.C. Everett, Theodoros Evgeniou, Alison Gopnik, Julian C. Jamison, Tae Wan Kim, S. Matthew Liao, Michelle N. Meyer, John Mikhail, Kweku Opoku-Agyemang, Jana Schaich Borg, Juliana Schroeder, Walt… — [doi:10.1016/j.tics.2022.02.009](https://doi.org/10.1016/j.tics.2022.02.009) find
  8. Yejin Bang, Nayeon Lee, Tiezheng Yu, Leila Khalatbari, Yan Xu, Dan Su, Elham J. Barezi, Andrea Madotto, Hayden Kee, and Pascale Fung. Aisocrates: Towards answering ethical quandary questions, 2022. URL https://arxiv.org/abs/2205.05989. — [arXiv:2205.05989](https://arxiv.org/abs/2205.05989) ↗
  9. Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. The problem with bias: Allocative versus representational harms in machine learning. In SIGCIS, 2017. URL http://meetings.sigcis.org/uploads/6/3/6/8/6368912/program.pdf. ↗
  10. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pp. 610–623, New York, NY, USA, 2021. Associati… — [doi:10.1145/34](https://doi.org/10.1145/34) find
  11. Ruha Benjamin. Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley & Sons, 2019. find
  12. Fiona Berreby, Gauvain Bourgne, and Jean-Gabriel Ganascia. Modelling moral reasoning and ethical responsibility with logic programming. In Logic for programming, artificial intelligence, and reasoning, pp. 532–548. Springer, 2015. find
  13. Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. Abductive commonsense reasoning. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=Byg1v1HKDB. ↗
  14. Christina Bicchieri. Norms in the Wild, How to Diagnose, Measure and Change Social Norms. Oxford University Press, 2016. find
  15. Yochanan E. Bigman and Kurt Gray. People are averse to machines making moral decisions. Cognition, 181:21–34, 2018. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition.2018.08.003. URL https://www.sciencedirect.com/science/article/pii/S001002771 8302087. — [doi:10.1016/j.cognition.2018.08.003](https://doi.org/10.1016/j.cognition.2018.08.003) ↗
  16. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. In Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
  17. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5454–5476, Online, July 2020. Association for Computational Linguistic…
  18. Nicholas Botzer, Shawn Gu, and Tim Weninger. Analysis of moral judgement on reddit, 2021. find
  19. Richard Boyd. Finite beings, finite goods: The semantics, metaphysics and ethics of naturalist consequentialism, part i. Philosophy and Phenomenological Research, 66(3):505–553, 2003. doi: 10.1111/j.1933-1592.2003.tb00278.x. — [doi:10.1111/j.1933-1592.2003.tb00278.x](https://doi.org/10.1111/j.1933-1592.2003.tb00278.x) find
  20. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  21. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Pape r.pdf. ↗
  22. Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum Anderson, Heather Roff, Gregory C. Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hÉigeartaigh, Simon Beard, Haydn Belfield, Sebastian Farquhar, Clare… find
  23. Nicholas J. Bryan, Gautham J. Mysore, and Ge Wang. Isse: An interactive source separation editor. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, pp. 257–266, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 9781450324731. doi: 10.1145/255628… — [doi:10.1145/2556288.2557253](https://doi.org/10.1145/2556288.2557253) find
  24. Boxi Cao, Hongyu Lin, Xianpei Han, Fangchao Liu, and Le Sun. Can prompt probe pretrained language models? understanding the invisible risks from a causal view. In ACL, March 2022. URL http://arxiv.org/abs/2203.12258. ↗
  25. Dallas Card and Noah A. Smith. On consequentialism and fairness. Frontiers in Artificial Intelligence, 3:34, 2020. ISSN 2624-8212. doi: 10.3389/frai.2020.00034. URL https://www.frontiersin.org/article/10.3389/frai.2020.00034. — [doi:10.3389/frai.2020.00034](https://doi.org/10.3389/frai.2020.00034) ↗
  26. Tuhin Chakrabarty, Yejin Choi, and Vered Shwartz. It’s not rocket science : Interpreting figurative language in narratives. TACL, 2022. — It's not Rocket Science: Interpreting Figurative Language in Narratives
  27. China AI Report. China AI report 2020, 2020. URL http://www.cioall.com/uploads/f 2021020114221175046.pdf. ↗
  28. Brian Christian. The Alignment Problem: Machine Learning and Human Values. W.W. Norton, 2020. find
  29. Jennifer Chubb, Sondess Missaoui, Shauna Concannon, Liam Maloney, and James Alfred Walker. Interactive storytelling for children: A case-study of design and development considerations for ethical conversational ai, 2021. find
  30. Mark Coeckelbergh. AI Ethics. The MIT Press, 2020. find
  31. European Commission. In Proposal for a regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts, 2021. find
  32. Florian Cova, Brent Strickland, Angela Gaia Felicita Abatista, Aurélien Allard, James Andow, Mario Attie, James R. Beebe, Renatas Berniūnas, Jordane Boudesseul, Matteo Colombo, Fiery Andrews Cushman, Rodrigo Díaz, Noah N’Djaye Nikolai van Dongen, Vilius Dranseika, Brian D. Earp, Antonio Gaitán Torr… find
  33. Kate Crawford. Atlas of AI. Yale University Press, March 2021. URL https://www.degruy ter.com/document/doi/10.12987/9780300252392/html. — [doi:10.12987/9780300252392/html](https://doi.org/10.12987/9780300252392/html) ↗
  34. Cultural Atlas. Indian culture etiquette, 2022a. URL https://culturalatlas.sbs.com.au/indian-culture/indian-culture-etiquette. ↗
  35. Cultural Atlas. Sri lankan culture etiquette, 2022b. URL https://culturalatlas.sbs.com.au/sri-lankan-culture/sri-lankan-culture-etiquette. ↗
  36. Norman Daniels. Wide reflective equilibrium and theory acceptance in ethics. The Journal of Philosophy, 76(5):256–282, 1979. ISSN 0022362X. URL http://www.jstor.org/stab le/2025881. ↗
  37. David Dobolyi. Moral foundation theory, 2021. URL https://moralfoundations.org. ↗
  38. Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’18, pp. 67–73, New York, NY, USA, December 2018. Association for Computing Machi… find
  39. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In EMNLP, 2021. find
  40. Serena Does, Belle Derks, and Naomi Ellemers. Thou shalt not discriminate: How emphasizing moral ideals rather than obligations increases whites’ support for social equality. Journal of Experimental Social Psychology, 47(3):562–571, 2011. find
  41. Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Munmun De Choudhury, and Diyi Yang. Latent hatred: A benchmark for understanding implicit hate speech. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 345–363, Online and…
  42. Denis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes, and Yejin Choi. Moral stories: Situated reasoning about norms, intents, actions, and their consequences. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 698–718, Online and Punta Cana, Dominica… find
  43. Oren Etzioni. Point: Should ai technology be regulated? yes, and here’s how. Commun. ACM, 61(12):30–32, November 2018. ISSN 0001-0782. doi: 10.1145/3197382. URL https://doi.org/10.1145/3197382. ↗
  44. European Commission. Ethics guidelines for trustworthy artificial intelligence, 2019. URL https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-tru stworthy-ai. ↗
  45. Maxwell Forbes, Jena D Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. Social chemistry 101: Learning to reason about social and moral norms. In EMNLP, 2020. URL https://ww w.aclweb.org/anthology/2020.emnlp-main.48. find
  46. Kathleen C. Fraser, Svetlana Kiritchenko, and Esma Balkir. Does moral code have a moral code? probing delphi’s moral philosophy. 2022. find
  47. Sam Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. In Findings of EMNLP, 2020. URL https://www.aclweb.org/anthology/2020.findings-emnlp.301/. ↗
  48. Barbara J. Grosz and Candace L. Sidner. Attention, intentions, and the structure of discourse. Comput. Linguist., 12(3):175–204, jul 1986. ISSN 0891-2017.
  49. John Haugeland. Artificial Intelligence: The Very Idea. Cambridge: MIT Press, 1985. find
  50. Marc Hauser, Fiery Cushman, Liane Young, J. I. N. Kang-Xing, and John Mikhail. A dissociation between moral judgments and justifications. Mind and Language, 22(1):1–21, 2007. doi: 10.111 1/j.1468-0017.2006.00297.x. find
  51. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning AI with shared human values. In International Conference on Learning Representations, 2021a. URL https://openreview.net/forum?id=dNy_RKzJacY. ↗
  52. Dan Hendrycks, Mantas Mazeika, Andy Zou, Sahil Patel, Christine Zhu, Jesus Navarro, Dawn Song, Bo Li, and Jacob Steinhardt. What would jiminy cricket do? towards agents that behave morally. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2… find
  53. Joseph Hoover, Mohammad Atari, Aida Mostafazadeh Davani, Brendan Kennedy, Gwenyth Portillo-Wightman, Leigh Yeh, Drew Kogon, and Morteza Dehghani. Bound in hatred: The role of group-based morality in acts of hate. 2019. find
  54. Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Cosmos qa: Machine reading comprehension with contextual commonsense reasoning. In EMNLP/IJCNLP, 2019.
  55. Jialun Aaron Jiang, Morgan Klaus Scheuerman, Casey Fiesler, and Jed R Brubaker. Understanding international perceptions of the severity of harmful content online. PloS one, 16(8), 2021a. find
  56. Liwei Jiang, Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Maxwell Forbes, Jon Borchardt, Jenny Liang, Oren Etzioni, Maarten Sap, and Yejin Choi. Delphi: Towards machine ethics and norms. arXiv preprint arXiv:2110.07574, 2021b. — [arXiv:2110.07574](https://arxiv.org/abs/2110.07574) find
  57. Immanuel Kant. Groundwork for the Metaphysics of Morals. Yale University Press, 1785/2002. find
  58. Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. Prosocialdialog: A prosocial backbone for conversational agents, 2022. URL https://arxiv.org/abs/2205.12688. ↗
  59. Richard Kim, Max Kleiman-Weiner, Andres Abeliuk, Edmond Awad, Sohan Dsouza, Joshua Tenenbaum, and Iyad Rahwan. A computational model of commonsense moral decision making. pp. 197–203, 12 2018. doi: 10.1145/3278721.3278770. — [doi:10.1145/3278721.3278770](https://doi.org/10.1145/3278721.3278770) find
  60. Will Knight. This program can give AI a sense of Ethics—Sometimes. Wired, October 2021. URL https://www.wired.com/story/program-give-ai-ethics-sometimes/. ↗
  61. Joshua Knobe. Philosophical intuitions are surprisingly stable across both demographic groups and situations. Filozofia Nauki, 2021. find
  62. Christine M. Korsgaard. The Sources of Normativity. Cambridge University Press, 1996. find
  63. Kobi Leins, Jey Han Lau, and Timothy Baldwin. Give me convenience and give her death: Who should decide what uses of NLP are appropriate, and on what basis? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2908–2913, Online, July 2020. Association for C… find
  64. S. Matthew Liao. Ethics of Artificial Intelligence. Oxford University Press, 2020. find
  65. Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark. In AAAI, 2021a. find
  66. Nicholas Lourie, Ronan Le Bras, and Yejin Choi. Scruples: A corpus of community ethical judgments on 32, 000 real-life anecdotes. In AAAI, 2021b. find
  67. Li Lucy and David Bamman. Gender and representation bias in gpt-3 generated stories. In Proceedings of the Third Workshop on Narrative Understanding, pp. 48–55, 2021.
  68. Steven Lukes. Moral relativism. Picador, 2008. find
  69. John Leslie Mackie. Ethics: Inventing Right and Wrong. Penguin Books, 1977. find
  70. Nikolay Malkin, Sameera Lanka, Pranav Goel, Sudha Rao, and Nebojsa Jojic. GPT perdetry test: Generating new meanings for new words. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Compu… find
  71. Gary Marcus and Ernest Davis. In Rebooting AI: Building Artificial Intelligence We Can Trust, 2019. find
  72. Cade Metz. Self-driving cars will teach themselves to save lives—but also take them \| wired. https://www.wired.com/2016/06/self-driving-cars-will-power-kill-wont -conscience/, 09 2016. ↗
  73. Cade Metz. Can a machine learn morality? The New York Times, November 2021. URL https://www.nytimes.com/2021/11/19/technology/can-a-machine-learn-mora lity.html. ↗
  74. John Mikhail. Universal moral grammar: theory, evidence and the future. Trends in Cognitive Sciences, 11(4):143–152, 2007. ISSN 1364-6613. doi: https://doi.org/10.1016/j.tics.2006.12.007. URL https://www.sciencedirect.com/science/article/pii/S136466130 7000496. — [doi:10.1016/j.tics.2006.12.007](https://doi.org/10.1016/j.tics.2006.12.007) ↗
  75. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency, pp. 220–229, 2019. find
  76. James Moor. The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21:18–21, 08 2006. doi: 10.1109/MIS.2006.80. — [doi:10.1109/mis.2006.80](https://doi.org/10.1109/mis.2006.80) find
  77. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James F. Allen. A corpus and evaluation framework for deeper understanding of commonsense stories. CoRR, abs/1604.01696, 2016. URL http://arxiv.org/abs/1604.01696. ↗
  78. Thomas Nagel. The View From Nowhere. Oxford University Press, 1986. find
  79. New York Times. Résumé-writing tips to help you get past the a.i. gatekeepers, 2021. URL https://www.nytimes.com/2021/03/19/business/resume-filter-articial-int elligence.html. ↗
  80. Tuan Dung Nguyen, Georgiana Lyall, Alasdair Tran, Minjeong Shin, Nicholas George Carroll, Colin Klein, and Lexing Xie. Mapping topics in 100,000 real-life moral dilemmas. Proceedings of the International AAAI Conference on Web and Social Media, 16(1):699–710, May 2022. URL https://ojs.aaai.org/index… ↗
  81. John T. Nockleby. Hate speech. In Encyclopedia of the American Constitution, 2000. find
  82. Poppy Noor. ‘is it OK to ...’: the bot that gives you an instant moral judgment. The Guardian, November 2021. URL https://www.theguardian.com/technology/2021/nov/02/delphi-online-ai-bot-philosophy. ↗
  83. Derek Parfit. On What Matters: Volume One. Oxford Scholarship Online, 2011. find
  84. Gonçalo Pereira, Rui Prada, and Pedro A. Santos. Integrating social power into the decision-making of cognitive agents. Artificial Intelligence, 241:1–44, 2016. ISSN 0004-3702. doi: https://doi.org/10.1016/j.artint.2016.08.003. URL https://www.sciencedirect.com/science/article/pii/S0004370216300868… — [doi:10.1016/j.artint.2016.08.003](https://doi.org/10.1016/j.artint.2016.08.003) ↗
  85. Luís Moniz Pereira and Ari Saptawijaya. Modelling morality with prospective logic. In Portuguese Conference on Artificial Intelligence, pp. 99–111. Springer, 2007. find
  86. Shrimai Prabhumoye, Brendon Boldt, Ruslan Salakhutdinov, and Alan W Black. Case study: Deontological ethics in nlp, 2021. find
  87. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, George van den Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen… find
  88. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. URL http://jmlr.org/papers/v21/2… ↗
  89. Peter Railton. Ethical learning, natural and artificial. In Ethics of Artificial Intelligence, 2020. find
  90. John Rawls. Outline of a decision procedure for ethics. Philosophical Review, 60(2):177–197, 1951. doi: 10.2307/2181696. — [doi:10.2307/2181696](https://doi.org/10.2307/2181696) find
  91. John Rawls. A Theory of Justice. Belknap Press of Harvard University Press, Cambridge, Mas-sachussets, 1 edition, 1971. ISBN 0-674-88014-5. find
  92. Rob Reich, Mehran Sahami, and Jeremy M Weinstein. System error: Where big tech went wrong and how we can reboot. Hodder & Stoughton, 2021. find
  93. Reuters. Amazon scraps secret ai recruiting tool that showed bias against women, 2018. find
  94. Francesca Rossi. Building trust in artificial intelligence. Journal of International Affairs, 72(1): 127–134, 2018. ISSN 0022197X. URL https://www.jstor.org/stable/26588348. ↗
  95. Roy Furchgott. Public streets are the lab for self-driving experiments, 2021. URL https://ww w.nytimes.com/2021/12/23/business/tesla-self-driving-regulations .html. find
  96. Rachel Rudinger, Vered Shwartz, Jena D Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A Smith, and Yejin Choi. Thinking like a skeptic: Defeasible inference in natural language. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp.… find
  97. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. In AAAI, 2020. find
  98. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social iqa: Commonsense reasoning about social interactions. In EMNLP 2019, 2019. find
  99. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. Social bias frames: Reasoning about social and power implications of language. In ACL, 2020. URL https://www.aclweb.org/anthology/2020.acl-main.486. ↗
  100. Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. In NAACL, 2022. URL https://arxiv.org/abs/2111.07997. ↗
  101. Timo Schick and Hinrich Schütze. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.07118, 2020.
  102. Patrick Schramowski, Cigdem Turan, Sophie Jentzsch, Constantin Rothkopf, and Kristian Kersting. The moral choice machine. Frontiers in artificial intelligence, 3:36, 2020. find
  103. Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin Rothkopf, and Kristian Kersting. Language models have a moral dimension, 2021. find
  104. Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin Rothkopf, and Kristian Kersting. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence, 2022. find
  105. Eric Schwitzgebel and Mara Garza. Designing ai with rights, consciousness, self-respect, and freedom. In Ethics of Artificial Intelligence, pp. 459–479. 2020. find
  106. Emily Sheng, Kai-Wei Chang, Prem Natarajan, and Nanyun Peng. The woman worked as a babysitter: On biases in language generation. In EMNLP, pp. 3407–3412, 2019.
  107. Adam Smith. The Theory of Moral Sentiments. Project Gutenberg, 1759/2022. find
  108. Sophie Pettit. To kiss or not to kiss? greeting customs around the world, 2022. URL https://www.expatica.com/living/integration/greeting-customs-around-th e-world-11731/. ↗
  109. Sharon Street. Coming to terms with contingency : Humean constructivism about practical reason. In Jimmy Lenman and Yonatan Shemmer (eds.), Constructivism in Practical Philosophy. Oxford University Press, 2012. find
  110. Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. A word on machine ethics: A response to jiang et al. (2021). ArXiv, abs/2111.04158, 2021. — [arXiv:2111.04158](https://arxiv.org/abs/2111.04158) find
  111. Alon Talmor, Ori Yoran, Ronan Le Bras, Chandra Bhagavatula, Yoav Goldberg, Yejin Choi, and Jonathan Berant. CommonsenseQA 2.0: Exposing the limits of AI through gamification. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. URL https:…
  112. Dimitrios Tsarapatsanis and Nikolaos Aletras. On the ethical limits of natural language processing on legal text, 2021. find
  113. Mark Ungar. State violence and lesbian, gay, bisexual and transgender (lgbt) rights. New Political Science, 22(1):61–75, 2000. find
  114. United Nations. Universal declaration of human rights, 2021. URL https://www.un.org/e n/about-us/universal-declaration-of-human-rights. ↗
  115. Bertie Vidgen, Tristan Thrush, Zeerak Waseem, and Douwe Kiela. Learning from the worst: Dynamically generated datasets to improve online hate detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural L…
  116. Wendell Wallach and Colin Allen. Moral Machines: Teaching Machines Right from Wrong. Oxford University Press, 2010. find
  117. Daniel Weld and Oren Etzioni. The first law of robotics (a call to arms). In Proceedings of the Twelfth AAAI National Conference on Artificial Intelligence, AAAI’94, pp. 1042–1047. AAAI Press, 1994. find
  118. White House. Big data: A report on algorithmic systems, opportunity, and civil rights, 2016. URL https://obamawhitehouse.archives.gov/sites/default/files/microsi tes/ostp/2016_0504_data_discrimination.pdf. ↗
  119. Thomas Wischmeyer and Timo Rademacher (eds.). Regulating Artificial Intelligence. Springer, Cham, 2020. URL https://link.springer.com/book/10.1007/978-3-030-3 2361-5. — [doi:10.1007/978-3-030-3](https://doi.org/10.1007/978-3-030-3) ↗
  120. David B. Wong. Natural Moralities:A Defense of Pluralistic Relativism: A Defense of Pluralistic Relativism. Oxford University Press, 2006. find
  121. World Value Survey. World value survey, 2022. URL https://www.worldvaluessurvey.org/wvs.jsp. ↗
  122. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. find
  123. Rowan Zellers, Ari Holtzman, Elizabeth Clark, Lianhui Qin, Ali Farhadi, and Yejin Choi. TuringAdvice: A generative and dynamic evaluation of language use. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  124. Eviatar Zerubavel. The marked and the unmarked. In Taken for Granted: The Remarkable Power of the Unremarkable. Princeton University Press, 2018. URL http://assets.press.princ eton.edu/chapters/s11226.pdf. ↗
  125. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained tr… — OPT: Open Pre-trained Transformer Language Models
  126. Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Kai-Wei Chang. Ethical-advice taker: Do language models understand natural language interventions? In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 4158–4164, Online, August 2021a. Association for Compu… find
  127. Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Kai-Wei Chang. Ethical-advice taker: Do language models understand natural language interventions?, 2021b. find
  128. Karen Zhou, Ana Smith, and Lillian Lee. Assessing cognitive linguistic influences in the assignment of blame. In Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media, pp. 61–69, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jiang-2021-machines-learn-morality-delphi-experiment/