Elamur Research⌕
☰
home / papers / 2023 / Large Language Models Understand and Can be Enhanced by Emotional Stimuli

Large Language Models Understand and Can be Enhanced by Emotional Stimuli

Paper · 2023 · Cheng Li, Jindong Wang, Yixuan Zhang et al.

Summary

This study asks whether large language models react to psychologically grounded emotional cues the way humans do, and whether such cues can improve their output. The authors propose EmotionPrompt, which appends one of 11 short motivational or pressuring sentences (EP01–EP11) to an ordinary instruction. The sentences draw on research about self-monitoring, Social Cognitive theory and cognitive emotion regulation. EmotionPrompt is tested automatically on 45 tasks (24 from Instruction Induction and 21 selected from BIG-Bench) using Flan-T5-Large, Vicuna, Llama 2, BLOOM, ChatGPT and GPT-4. A separate evaluation with 106 human participants rates open-ended GPT-4 answers. Adding a stimulus yields an 8.00% relative gain on Instruction Induction and 115% on BIG-Bench. Human raters report an average 10.9% improvement across performance, truthfulness and responsibility. An input-attention analysis suggests the appended sentences receive notable weight and reinforce the original instruction's representation. The result matters because it is a free, training-less prompting technique, and it invites further work connecting psychology with LLM behaviour.

Key points

  • Setup: 6 LLMs, 45 tasks and 11 stimuli. EP06 is a compound of EP01, EP02 and EP03. One example stimulus tells the model the task matters for the user's career.
  • Best stimuli: EP02 performs best on Instruction Induction, beating the weakest stimulus by 6.06%. EP06 performs best on BIG-Bench. Combining several stimuli can raise scores further.
  • Relative Gain by model (zero-shot Instruction Induction): Vicuna 9.58, Llama 2 6.00, ChatGPT 4.32, GPT-4 0.85, BLOOM 0.51, Flan-T5-Large 0.28.
    • The authors attribute the small gains of strong models partly to their high baselines.
    • Vicuna and Llama 2 share the same 13B size but show different gains, which hints that training strategy (SFT/RLHF) matters.
  • Temperature ablation (8 tasks, 5 temperatures): the benefit grows as temperature rises. EmotionPrompt outputs also vary less with temperature than vanilla prompts, suggesting better robustness.
  • Open question: in humans, emotional stimuli are not thought to directly improve reasoning ability. The authors leave this human–LLM divergence for future research.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States240 works · 123,537 citations🎓Hong Kong University of Science and TechnologyHong Kong · China58 works · 23,058 citations🎓Beijing Normal UniversityBeijing · China6 works · 3,107 citations🔬Institute of SoftwareBeijing · China10 works · 4,807 citations🎓William & MaryWilliamsburg · United States2 works · 340 citations

Also on the byline: CAS Software.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 1,343 worksSentiment & emotion · 608 works

Cite

Cite as Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang and Xing Xie. 2023. Large Language Models Understand and Can be Enhanced by Emotional Stimuli. arXiv:2307.11760.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2307.11760 · arXiv 2307.11760

@misc{li2023large,
  title = {{Large Language Models Understand and Can be Enhanced by Emotional Stimuli}},
  author = {Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang and Xing Xie},
  year = {2023},
  doi = {10.48550/arxiv.2307.11760},
  eprint = {2307.11760},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2307.11760}
}

References

40 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Albert Bandura. Health promotion from the perspective of social cognitive theory. Psychology and health, 13(4):623–649, 1998. find
  2. Albert Bandura. On the functional properties of perceived self-efficacy revisited, 2012. find
  3. Albert Bandura. Health promotion from the perspective of social cognitive theory, 2013. find
  4. Albert Bandura and Edwin A Locke. Negative self-efficacy and goal effects revisited. Journal of applied psychology, 88(1):87, 2003. find
  5. Urszula Barańczuk. The five factor model of personality and emotion regulation: A meta-analysis. Personality and Individual Differences, 139:217–227, 2019. find
  6. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
  7. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang…
  8. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. A survey on in-context learning, 2023.
  9. Susan T Fiske and Shelley E Taylor. Social cognition. Mcgraw-Hill Book Company, 1991. find
  10. Barbara L Fredrickson. The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American psychologist, 56(3):218, 2001. find
  11. Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes, 2023. find
  12. Peter A Heslin and Ute-Christine Klehe. Self-efficacy. Encyclopedia Of Industrial/Organizational Psychology, SG Rogelberg, ed, 2:705–708, 2006. find
  13. Or Honovich, Uri Shaham, Samuel R. Bowman, and Omer Levy. Instruction induction: From few examples to natural language task descriptions, 2022. find
  14. William Ickes, Renee Holloway, Linda L Stinson, and Tiffany Graham Hoodenpyle. Self-monitoring in social interaction: The centrality of self-affect. Journal of personality, 74(3):659–684, 2006. find
  15. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916, 2022.
  16. Sander L Koole. The psychology of emotion regulation: An integrative review. Cognition and emotion, 23(1):4–41, 2009. find
  17. Richard S Lazarus. How emotions influence performance in competitive sports. The sport psychologist, 14(3):229–252, 2000. find
  18. Jennifer S Lerner, Ye Li, Piercarlo Valdesolo, and Karim S Kassam. Emotion and decision making. Annual review of psychology, 66:799–823, 2015. find
  19. Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2021.
  20. Aleksandra Luszczynska and Ralf Schwarzer. Social cognitive theory. Fac Health Sci Publ, pages 225–51, 2015. find
  21. Marios Miltiadou and Wilhelmina C Savenye. Applying social cognitive constructs of motivation to enhance student success in online distance education. AACE Review (formerly AACE Journal), 11(1):78–95, 2003. find
  22. Arne Öhman, Anders Flykt, and Francisco Esteves. Emotion drives attention: detecting the snake in the grass. Journal of experimental psychology: general, 130(3):466, 2001. find
  23. OpenAI. Chatgpt. https://chat.openai.com/, 2023. ↗
  24. OpenAI. Gpt-4 technical report, 2023.
  25. Reinhard Pekrun, Thomas Goetz, Wolfram Titz, and Raymond P Perry. Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational psychologist, 37(2):91–105, 2002. find
  26. James A Russell. Core affect and the psychological construction of emotion. Psychological review, 110(1):145, 2003. find
  27. Peter Salovey, John D Mayer, David Caruso, and Seung Hee Yoo. The positive psychology of emotional intelligence. The Oxford handbood of positive psychology, 2009. find
  28. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighof… find
  29. Dale H Schunk and Maria K DiBenedetto. Self-efficacy and human motivation. Advances in motivation science, 8:153–179, 2021. find
  30. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Al…
  31. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them, 2022.
  32. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… find
  33. Xuena Wang, Xueting Li, Zi Yin, Yue Wu, and Liu Jia. Emotional intelligence of large language models. arXiv preprint arXiv:2307.09042, 2023. — [arXiv:2307.09042](https://arxiv.org/abs/2307.09042) find
  34. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 2022.
  35. Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, Ji Zhang, Chao Peng, Fei Huang, and Jingren Zhou. Cvalues: Measuring the values of chinese large language models from safety to responsibility, 2023.
  36. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. CoRR, abs/2305.10601, 2023. find
  37. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.
  38. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. find
  39. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers, 2023.
  40. Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, et al. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv preprint arXiv:2306.04528, 2023.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2023-large-language-models-understand-enhanced/