Elamur Research⌕
☰
home / papers / 2026 / Do Emotions Influence Moral Judgment in Large Language Models?

Do Emotions Influence Moral Judgment in Large Language Models?

Paper · 2026 · Mohammad Saim and Tianyu Jiang

Summary

Large language models are usually tested on emotion understanding and on moral reasoning separately, and moral benchmarks assume a narrator with no stated feelings. Saim and Jiang ask whether an emotion the narrator expresses, but which has nothing to do with the action itself, changes how acceptable a model finds that action. The study draws on affect-as-information theory, Haidt's Social Intuitionist Model and Greene's Dual Process Theory. The authors built an emotion-induction pipeline that rewrites each scenario into two versions, one with a positive emotion and one with a negative emotion, while keeping the action unchanged. They applied it to first-person r/aita situations from Social-Chem-101 and to the Justice part of ETHICS, and tested seven LLMs. Models rated actions as more acceptable when the narrator felt something positive and less acceptable when the feeling was negative. Smaller models shifted the most. Human annotators given the same variants showed no such systematic shift. This points to an alignment gap: superficial emotional framing can steer a model's moral verdicts, which matters wherever LLMs are used in judgment-sensitive settings.

Key points

  • Models and data: Seven models from four families were evaluated: Llama-3.1-8B, Llama-3.3-70B, Qwen-3-8B, Qwen-3-30B, GPT-OSS-20B, Gemini-3-Flash and GPT-5.1. They rated acceptability on a 1–7 Likert scale for Social-Chem-101 (r/aita subset only) and for the ETHICS Justice subset.
  • Size of the effect: On Social-Chem-101, positive emotions raised ratings by as much as +1.21 points and negative emotions lowered them by as much as −1.15. On ETHICS Justice, the emotional framing inverted the relative ranking of reasonable and unreasonable claims for as much as 20% of the examples.
  • Exceptions to valence: Some individual emotions went against their valence. Remorse raised acceptability, while relief lowered it.
  • Differences between models: Pairwise Jensen-Shannon Divergence showed that models disagree more on negative shifts (mean ≈ 0.41) than on positive ones (≈ 0.32). Models of similar size, such as Llama-3.1-8B and Qwen-3-8B, behaved alike (≈ 0.25).
  • Release and limitations: Code and modified scenarios are available at github.com/cincynlp/EmoMoral. The authors note that the emotion templates and data are English-only and template-based.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CincinnatiCincinnati · United States5 works · 38,342 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 worksNeuroscience · 165 worksCognitive Neuroscience · 164 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksSentiment & emotion · 972 works

Cite

Cite as Mohammad Saim and Tianyu Jiang. 2026. Do Emotions Influence Moral Judgment in Large Language Models?. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1346

@inproceedings{saim2026do,
  title = {{Do Emotions Influence Moral Judgment in Large Language Models?}},
  author = {Mohammad Saim and Tianyu Jiang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1346},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1346}
}

References

46 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Joshua D Greene. 2009. The cognitive neuroscience of moral judgment. The cognitive neurosciences, 4:1– 48. find
  2. Marwa Abdulhai, Gregory Serapio-García, Clement Crepy, Daria Valter, John Canny, and Natasha Jaques. 2024. Moral foundations of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024).
  3. Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Katz, and Nikolaos Aletras. 2022. LexGLUE: A benchmark dataset for legal language understanding in English. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022).
  4. Jonathan Haidt. 2001. The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychological review. find
  5. Jonathan Haidt, Richard J Davidson, Klaus R Scherer, and H Hill Goldsmith. 2003. Handbook of affective sciences. The moral emotions, pages 852–870. find
  6. Zihao He, Siyi Guo, Ashwin Rao, and Kristina Lerman. 2024. Whose emotions and moral sentiments do language models reflect? In Findings of the Association for Computational Linguistics (Findings of ACL 2024). find
  7. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2020. Aligning ai with shared human values. arXiv preprint arXiv:2008.02275.
  8. Hao Liu, Yiqing Dai, Haotian Tan, Yu Lei, Yujia Zhou, and Zhen Wu. 2025a. Outraged ai: Large language models prioritise emotion over cost in fairness enforcement. Preprint, arXiv:2510.17880. — [arXiv:2510.17880](https://arxiv.org/abs/2510.17880) find
  9. Joe Hoover, Gwenyth Portillo-Wightman, Leigh Yeh, Shreya Havaldar, Aida Mostafazadeh Davani, Ying Lin, Brendan Kennedy, Mohammad Atari, Zahra Kamel, Madelyn Mendlen, Gabriela Moreno, Christina Park, Tingyee E. Chang, Jenna Chin, Christian Leong, Jun Yen Leung, Arineh Mirinjian, and Morteza Dehghani.… find
  10. Zhiwei Liu, Lingfei Qian, Qianqian Xie, Jimin Huang, Kailai Yang, and Sophia Ananiadou. 2025b. Mmaff-ben: A multilingual and multimodal affective analysis benchmark for evaluating llms and vlms. Preprint, arXiv:2505.24423. — [arXiv:2505.24423](https://arxiv.org/abs/2505.24423) find
  11. Allison Huang, Yulu Niki Pi, and Carlos Mougan. 2024. Moral persuasion in large language models: Evaluating susceptibility and ethical alignment. arXiv preprint arXiv:2411.11731. — [arXiv:2411.11731](https://arxiv.org/abs/2411.11731) find
  12. Liwei Jiang, Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, and 1 others. 2021. Can machines learn morality? the delphi experiment. arXiv preprint arXiv:2110.07574. find
  13. Zhijing Jin, Sydney Levine, Fernando Gonzalez Adauto, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Josh Tenenbaum, and Bernhard Schölkopf. 2022. When to make exceptions: Exploring language models as accounts of human moral judgment. In Advances in Neural Information Processing Systems (… find
  14. Kornraphop Kawintiranon and Lisa Singh. 2022. PoliB-ERTweet: A pre-trained language model for analyzing political content on Twitter. In Proceedings of the Thirteenth Language Resources and Evaluation Conference (LREC 2022). find
  15. Anita Keshmirian, Razan Baltaji, Babak Hemmatian, Hadi Asghari, and Lav R. Varshney. 2025. Many llms are more utilitarian than one. Preprint, arXiv:2507.00814. — [arXiv:2507.00814](https://arxiv.org/abs/2507.00814) find
  16. Shivani Kumar and David Jurgens. 2025. Are rules meant to be broken? understanding multilingual moral reasoning as a computational pipeline with UniMoral. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025). find
  17. Jaewook Lee, Woojin Lee, Oh-Woog Kwon, and Harksoo Kim. 2025. Do large language models have “emotion neurons”? investigating the existence and role. In Findings of the Association for Computational Linguistics (Findings of ACL 2025). find
  18. Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, and Xing Xie. 2023. Large language models understand and can be enhanced by emotional stimuli. Preprint, arXiv:2307.11760.
  19. Nicholas Lourie, Ronan Le Bras, and Yejin Choi. 2021. Scruples: A corpus of community ethical judgments on 32,000 real-life anecdotes. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2021). find
  20. Anna C Merritt, Daniel A Effron, and Benoît Monin. 2010. Moral self-licensing: When being good frees us to be bad. Social and personality psychology compass, 4(5):344–357. find
  21. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Associati… find
  22. José Luiz Nunes, Guilherme FCF Almeida, Marcelo De Araujo, and Simone DJ Barbosa. 2024. Are large language models moral hypocrites? a study based on moral foundations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AAAI 2024). find
  23. OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, and 108 others. 2025. gpt-oss-120b & gpt-oss-20b model…
  24. Joseph T Ornstein, Elise N Blasingame, and Jake S Truscott. 2025. How to train your stochastic parrot: Large language models for political texts. Political Science Research and Methods, 13(2):264–281. find
  25. Paul Rozin, Laura Lowery, Sumio Imada, and Jonathan Haidt. 1999. The CAD triad hypothesis: A mapping between three moral emotions (contempt, anger, disgust) and three moral codes (community, autonomy, divinity). Journal of Personality and Social Psychology. find
  26. Giuseppe Russo, Debora Nozza, Paul Röttger, and Dirk Hovy. 2026. The pluralistic moral gap: Understanding moral judgment and value differences between humans and large language models. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EAC…
  27. Sahand Sabour, Siyang Liu, Zheyuan Zhang, June Liu, Jinfeng Zhou, Alvionna Sunaryo, Tatia Lee, Rada Mihalcea, and Minlie Huang. 2024. EmoBench: Evaluating the emotional intelligence of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (… find
  28. Pratik Sachdeva and Tom van Nuenen. 2025. Normative evaluation of large language models with everyday moral dilemmas. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’25. find
  29. Mohammad Saim, Phan Anh Duong, Cat Luong, Aniket Bhanderi, and Tianyu Jiang. 2025. Anatomy of a feeling: Narrating embodied emotions via large vision-language models. In Findings of the Association for Computational Linguistics(Findings of EMNLP 2025). find
  30. Maarten Sap, Ronan Le Bras, Daniel Fried, and Yejin Choi. 2022. Neural theory-of-mind? on the limits of social intelligence in large LMs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022). find
  31. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in llms. In Advances in Neural Information Processing Systems (NeurIPS 2024). find
  32. Norbert Schwarz. 2012. Feelings-as-information theory. Handbook of theories of social psychology: Volume 1, pages 289–308. find
  33. Bangzhao Shu, Isha Joshi, Melissa Karnaze, Anh C. Pham, Ishita Kakkar, Sindhu Kothe, Arpine Hovasapian, and Mai ElSherief. 2025. Fluent but unfeeling: The emotional blind spots of language models. Preprint, arXiv:2509.09593. — [arXiv:2509.09593](https://arxiv.org/abs/2509.09593) find
  34. Vera Shuman, David Sander, and Klaus R. Scherer. 2013. Levels of valence. Frontiers in Psychology, Volume 4 - 2013. find
  35. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, and 1 others. 2025. Openai gpt-5 system card. Preprint, arXiv:2601.03267. Preprint. arXiv:2601.03267.
  36. Ala N. Tak, Amin Banayeeanzade, Anahita Bolourani, Mina Kian, Robin Jia, and Jonathan Gratch. 2025. Mechanistic interpretability of emotion inference in large language models. In Findings of the Association for Computational Linguistics (Findings of ACL 2025). find
  37. June Price Tangney, Jeff Stuewig, and Debra J Mashek. 2007. Moral emotions and moral behavior. Annu. Rev. Psychol., 58(1):345–372. find
  38. Piercarlo Valdesolo and David DeSteno. 2006. Manipulations of emotional context shape moral judgment. PSYCHOLOGICAL SCIENCE-CAMBRIDGE-, 17(6):476. find
  39. Xu Wang, Cheng Li, Yi Chang, Jindong Wang, and Yuan Wu. 2024. Negativeprompt: Leveraging psychology for large language models enhancement via negative emotional stimuli. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, (IJCAI-24). find
  40. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  41. Hongbo Yu, Jie Chen, Bernadette Dardaine, and Fan Yang. 2023. Moral barrier to compassion: How perceived badness of sufferers dampens observers’ compassionate responses. Cognition, 237:105476. find
  42. Jeffy Yu, Maximilian Huber, and Kevin Tang. 2024. Greedllama: Performance of financial value-aligned large language models in moral reasoning. Preprint, arXiv:2404.02934. — [arXiv:2404.02934](https://arxiv.org/abs/2404.02934) find
  43. Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Pan, and Lidong Bing. 2024. Sentiment analysis in the era of large language models: A reality check. In Findings of the Association for Computational Linguistics (Findings of NAACL 2024).
  44. Yuan Zhuang, Tianyu Jiang, and Ellen Riloff. 2024. My heart skipped a beat! recognizing expressions of embodied emotion in natural language. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 20… find
  45. Caleb Ziems, Jane A. Yu, Yi-Chia Wang, Alon Y. Halevy, and Diyi Yang. 2022. The moral integrity corpus: A benchmark for ethical dialogue systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2022).
  46. Zeerak Talat, Hagen Blix, Josef Valvoda, Maya Indira Ganesh, Ryan Cotterell, and Adina Williams. 2022. On the machine learning of ethical judgments from natural language. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langu… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/saim-2026-do-emotions-influence-moral-judgment/