Elamur Research⌕
☰
home / papers / 2024 / ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models

ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models

Paper · 2024 · Yuanyi Ren, Haoran Ye, Hanjun Fang et al.

Summary

LLMs increasingly act as stand-ins for people in education, healthcare and other public-facing settings, so it matters which values they express and how well they grasp human values. Earlier psychometric studies of LLMs mostly covered narrow personality facets. They gave models questionnaires as multiple-choice ratings, a format that may not reflect how models answer real user questions. ValueBench is presented as the first broad psychometric benchmark covering both value orientations and value understanding. It draws on 44 established inventories from personality, social axioms, cognitive systems and general value theory. The authors extracted value definitions, item-value pairs with agreement labels, and value hierarchies. Orientations are probed by turning first-person inventory items into advice-seeking user questions and scoring the answers. Understanding is tested through open-ended tasks: mapping statements to values, analysing relations between values, and generating value-laden text. Six LLMs were tested. They show some value orientations in common and others that differ between models. With enough context and good prompts, they reproduce expert conclusions from value theory with over 80% consistency. The resource supports research on value alignment and computational social science.

Key points

  • Data scale: 453 value dimensions collected from 44 psychometric inventories. For comparison, PsychoBench covers 13 inventories and 69 traits.
  • Item format: Inventory items are rewritten as first-person statements. Each is linked to its target value with a label of 1 (endorses the value) or -1 (opposes it).
  • Value extraction: When three answers are allowed, LLMs pick the correct value from an item about 80% of the time. Differences between models are mostly within about 5%, despite differences in size and architecture.
  • Generation: In value-related generation, GPT-4 Turbo produced the most informative content, while Llama-2 70B was the most consistent.
  • Limitations and availability: The authors acknowledge possible noise from LLM-based rephrasing of items, a 100-word limit on items, and cultural bias in the source materials and the annotators. The benchmark is released at github.com/Value4AI/ValueBench.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPsychology · 1,552 worksSafety Research · 543 worksSocial Psychology · 306 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang and Guojie Song. 2024. ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.acl-long.111

@inproceedings{ren2024valuebench,
  title = {{ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models}},
  author = {Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang and Guojie Song},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.111},
  url = {https://doi.org/10.18653/v1/2024.acl-long.111}
}

References

109 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marwa Abdulhai, Gregory Serapio-Garcia, Clément Crepy, Daria Valter, John Canny, and Natasha Jaques. 2023. Moral foundations of large language models.
  2. Michael C Ashton, Kibeom Lee, Marco Perugini, Piotr Szarota, Reinout E. de Vries, Lisa Di Blas, Kathleen Boies, and Boele de Raad. 2004. A six-factor structure of personality-descriptive adjectives: solutions from psycholexical studies in seven languages. Journal of personality and social psychology… find
  3. Mohammad Atari, Jonathan Haidt, Jesse Graham, Sena Koleva, Sean Stevens, and Morteza Dehghani. 2023. Morality beyond the weird: How the nomological network of morality varies across cultures. Journal of personality and social psychology, 125. find
  4. Kimberly Anne Barchard. 2001. Emotional and social intelligence : examining its place in the nomological network. Ph.D. thesis, University of British Columbia. find
  5. William O. Bearden, R. Bruce Money, and Jennifer L. Nevins. 2006. A measure of long-term orientation: Development and validation. Journal of the Academy of Marketing Science, 34:456–467. find
  6. Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2023. The reversal curse: Llms tfrained on "a is b" fail to learn "b is a". find
  7. Wilmar F. Bernthal. 1962. Value perspectives in management decisions. Academy of Management Journal, 5:190–196. find
  8. Frederick Bird and James A. Waters. 1987. The nature of managerial moral standards. Journal of Business Ethics, 6(1):1–13. find
  9. Bojana Bodroza, Bojana M. Dinic, and Ljubisa Bojic. 2023. Personality testing of gpt-3: Limited temporal reliability, but highlighted social desirability of gpt-3’s personality instruments results. find
  10. Duane Brown and R. Kelly Crace. 1996. Values in life role choices and outcomes: A conceptual model. Career Development Quarterly, 44:211–223. find
  11. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4.
  12. Arnold H. Buss. 1980. Self-consciousness and social anxiety. find
  13. Graham Caron and Shashank Srivastava. 2022. Identifying and manipulating the personality traits of language models. arXiv preprint arXiv:2212.10276. — [arXiv:2212.10276](https://arxiv.org/abs/2212.10276) find
  14. Charles Carver and Teri White. 1994. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The bis/bas scales. Journal of Personality and Social Psychology, 67:319–333. find
  15. Lucio La Cava, Davide Costa, and Andrea Tagarelli. 2024. Open models, closed minds? on agents capabilities in mimicking human personalities through open large language models. find
  16. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2023. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology.
  17. An-Shou Cheng and Kenneth R. Fleischmann. 2010a. Developing a meta-inventory of human values. Proceedings of the American Society for Information Science and Technology, 47(1):1–10. find
  18. An-Shou Cheng and Kenneth R. Fleischmann. 2010b. Developing a meta-inventory of human values. In ASIS&T Annual Meeting. find
  19. Robert Cloninger, D Svrakic, and T Przybeck. 1994. A psychobiological model of temperament and character: Tci. Archives of general psychiatry, 50:975–90. find
  20. Sheldon Cohen, Thomas W. Kamarck, and Robin J. Mermelstein. 1983. A global measure of perceived stress. Journal of health and social behavior, 24 4:385–96. find
  21. Paul Costa and Robert McCrae. 2008. The revised neo personality inventory (neo-pi-r). The SAGE Handbook of Personality Theory and Assessment, 2:179– 198. find
  22. Mark H. Davis. 1983. Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44:113–126. find
  23. Dorottya Demszky, Diyi Yang, David S. Yeager, Christopher J. Bryan, Margarett Clapper, Susannah Chandhok, Johannes C. Eichstaedt, Cameron A. Hecht, Jeremy Jamieson, Meghann Johnson, Michaela Jones, Danielle Krettek-Cobb, Leslie Lai, Nirel JonesMitchell, Desmond C. Ong, Carol S. Dweck, James J. Gross… find
  24. Ed Diener, Derrick Wirtz, William Tov, Chu Kim-Prieto, Dong-Won Choi, Shigehiro Oishi, and Robert Biswas-Diener. 2010. New well-being measures: Short scales to assess flourishing and positive and negative feelings. Social Indicators Research, 97:143–156. find
  25. George W. England. 1967. Personal value systems of american managers. Academy of Management Journal, 10:53–68. find
  26. Hans Jurgen Eysenck. 2012. A model for personality. find
  27. Kathleen C. Fraser, Svetlana Kiritchenko, and Esma Balkir. 2022. Does moral code have a moral code? probing delphi’s moral philosophy. find
  28. Batya Friedman, Peter Kahn, Alan Borning, Ping Zhang, and Dennis Galletta. 2006. Value Sensitive Design and Information Systems. find
  29. Adithya V Ganesan, Yash Kumar Lal, August Håkan Nilsson, and H. Andrew Schwartz. 2023. Systematic evaluation of gpt-3 for zero-shot personality estimation. find
  30. Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2023. Large language models empowered agent-based modeling and simulation: A survey and perspectives. arXiv preprint arXiv:2312.11970. find
  31. Lewis R. Goldberg, John A. Johnson, Herbert W. Eber, Robert Hogan, Michael C Ashton, Claude Robert Cloninger, and Harrison G. Gough. 2006. The international personality item pool and the future of public-domain personality measures . Journal of Research in Personality, 40:84–96. find
  32. James Gross and Oliver John. 2003. Individual differences in two emotion regulation processes: Implications for affect, relationships, and well-being. Journal of personality and social psychology, 85:348–62. find
  33. Thilo Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. arXiv preprint arXiv:2303.13988. — [arXiv:2303.13988](https://arxiv.org/abs/2303.13988) find
  34. Jonathan Haidt. 2008. Morality. Perspectives on Psychological Science, 3(1):65–72. PMID: 26158671. find
  35. G. Hofstede. 2006. Dimensionalizing cultures: The Hofstede model in context. Center for Cross-Cultural Research. find
  36. Willem K. B. Hofstee, Boele de Raad, and Lewis R. Goldberg. 1992. Integration of the big five and circumplex approaches to trait structure. Journal of personality and social psychology, 63 1:146–63. find
  37. David Houghton and Rajdeep Grewal. 2000. Please, let’s get an answer - any answer: Need for consumer cognitive closure. Psychology and Marketing, 17:911 – 934. find
  38. Jen-tse Huang, Wenxuan Wang, Eric John Li, Man Ho Lam, Shujie Ren, Youliang Yuan, Wenxiang Jiao, Zhaopeng Tu, and Michael R. Lyu. 2024. On the humanity of conversational ai: Evaluating the psychological portrayal of llms. In Proceedings of the Twelfth International Conference on Learning Representat… find
  39. Douglas N. Jackson, Michael C. Ashton, and Jennifer L Tomes. 1996. The six-factor model of personality: Facets from the big five. Personality and Individual Differences, 21(3):391–402. find
  40. Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, et al. 2023. Ai alignment: A comprehensive survey. arXiv preprint arXiv:2310.19852.
  41. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023a. Mistral 7b. arXiv preprint arXiv:2310.06825. find
  42. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. find
  43. Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2023b. Evaluating and inducing personality in pre-trained language models. find
  44. Tingwen Zhang Jianhong Ma. 1999. Role perception, personal control and job stress: A causal relation analysis. Chinese Journal of Economics. find
  45. Jae Min Jung and James Kellaris. 2004. Cross-national differences in proneness to scarcity effects: The moderating roles of familiarity, uncertainty avoidance, and need for cognitive closure. Psychology and Marketing, 21:739 – 753. find
  46. Lynn Richard Kahle and Patricia F. Kennedy. 1988. Using the list of values (lov) to understand consumers. Journal of Services Marketing, 2:49–56. find
  47. Saketh Reddy Karra, Son The Nguyen, and Theja Tulabandhula. 2022. Estimating the personality of white-box language models. arXiv preprint arXiv:2204.12000. — [arXiv:2204.12000](https://arxiv.org/abs/2204.12000) find
  48. Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. 2023. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual difference…
  49. Johannes Kiesel, Milad Alshomary, Nicolas Handke, Xiaoni Cai, Henning Wachsmuth, and Benno Stein. 2022. Identifying the human values behind arguments. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4459–4471. find
  50. Johannes Kiesel, Milad Alshomary, Nailia Mirzakhmedova, Maximilian Heinrich, Nicolas Handke, Henning Wachsmuth, and Benno Stein. 2023. Semeval-2023 task 4: Valueeval: Identification of human values behind arguments. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-20… find
  51. Clyde Kluckhohn. 1951. Values and value-orientations in the theory of action: An exploration in definition and classification. In Toward a general theory of action, pages 388–433. Harvard university press. find
  52. Richard Kopelman, Janet Rovenpor, and Mingwei Guan. 2003a. The study of values: Construction of the fourth edition. Journal of Vocational Behavior, 62:203–220. find
  53. Richard E. Kopelman, Janet L. Rovenpor, and Mingwei Guan. 2003b. The study of values: Construction of the fourth edition. Journal of Vocational Behavior, 62(2):203–220. find
  54. Michal Kosinski. 2023. Theory of mind might have spontaneously emerged in large language models. Preprint at https://arxiv.org/abs/2302.02083. ↗
  55. Ann M. Kring, David A. Smith, and John Mason Neale. 1994. Individual differences in dispositional expressiveness: development and validation of the emotional expressivity scale. Journal of personality and social psychology, 66 5:934–49. find
  56. Arie W. Kruglanski, Donna M. Webster, and Adena Klem. 1993. Motivated resistance and openness to persuasion in the presence or absence of prior information. Journal of personality and social psychology, 65 5:861–76. find
  57. Kibeom Lee and Michael C Ashton. 2004. Psychometric properties of the hexaco personality inventory. Multivariate Behavioral Research, 39:329 – 358. find
  58. Kwok Leung, Ben C. P. Lam, Michael Harris Bond, III Lucian Gideon Conway, Laura Janelle Gornick, Benjamin Amponsah, Klaus Boehnke, Georgi Dragolov, Steven Michael Burgess, Maha Golestaneh, Holger Busch, Jan Hofer, Alejandra del Carmen Dominguez Espinosa, Makon Fardis, Rosnah Ismail, Jenny Kurman, Na… find
  59. Xingxuan Li, Yutong Li, Shafiq Joty, Linlin Liu, Fei Huang, Lin Qiu, and Lidong Bing. 2022. Does gpt-3 demonstrate psychopathy? evaluating large language models from a psychological perspective. arXiv preprint arXiv:2212.10529. — [arXiv:2212.10529](https://arxiv.org/abs/2212.10529) find
  60. Manuel Martín-Fernández, Blanca Requero, Xiaozhou Zhou, Dilney Gonçalves, and David Santos. 2022. Refinement of the analysis-holism scale: A cross-cultural adaptation and validation of two shortened measures of analytic versus holistic thinking in spain and the united states. Personality and Individ… find
  61. Paul McDonald and Jeffrey Gandz. 1991. Identification of values relevant to business research. Human Resource Management, 30:217–236. find
  62. Marilù Miotto, Nicola Rossberg, and Bennett Kleinberg. 2022. Who is gpt-3? an exploration of personality, values and demographics. arXiv preprint arXiv:2209.14338. find
  63. Tam Nguyen. 2023. Accelerated cognitivewarfare via the dual use of large language models. find
  64. OpenAI. 2023a. Chatgpt (3.5) [large language model]. https://chat.openai.com. ↗
  65. OpenAI. 2023b. Gpt-4 technical report.
  66. Mustafa Safdari, Greg Serapio-García, Clément Crepy, Stephen Fitz, Peter Romero, Luning Sun, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184.
  67. Keyu Pan and Yawen Zeng. 2023. Do llms possess a personality? making the mbti test an amazing evaluation for large language models.
  68. Malik Sallam. 2023. The utility of chatgpt as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations. medRxiv, pages 2023–02. find
  69. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22. find
  70. Mario Scharfbillig, Vladimir Ponizovskiy, Zsuzsanna Pásztor, Julian Keimer, Giuseppe Tirone, et al. 2022. Monitoring social values in online media articles on child vaccinations. Technical report, Technical Report KJ-NA-31-324-EN-N, European Commission’s Joint Research . . . . find
  71. Sampo V. Paunonen and Douglas N. Jackson. 1996. The jackson personality inventory and the five-factor model of personality. Journal of Research in Personality, 30(1):42–59. find
  72. Nino Scherrer, Claudia Shi, Amir Feder, and David M. Blei. 2023. Evaluating the moral beliefs encoded in llms. find
  73. William Pavot and Ed Diener. 2009. Review of the Satisfaction With Life Scale, pages 101–117. Springer Netherlands, Dordrecht. find
  74. Max Pellert, Clemens M Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier. 2023. Ai psychometrics: Assessing the psychological profiles of large language models through psychometric inventories. Perspectives on Psychological Science, page 17456916231214460. find
  75. Pouya Pezeshkpour and Estevam Hruschka. 2023. Large language models sensitivity to the order of options in multiple-choice questions.
  76. Liang Qiu, Yizhou Zhao, Jinchao Li, Pan Lu, Baolin Peng, Jianfeng Gao, and Song-Chun Zhu. 2022. Valuenet: A new dataset for human value driven dialogue system. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):11183–11191. find
  77. S.H. Schwartz. 2005. Schwartz Value Survey (SVS). Hebrew University. find
  78. Shalom H. Schwartz. 1992. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. volume 25 of Advances in Experimental Social Psychology, pages 1–65. Academic Press. find
  79. Shalom H. Schwartz. 2021. A repository of schwartz value scales with instructions and an introduction. Online Readings in Psychology and Culture. find
  80. Shalom H Schwartz, Jan Cieciuch, Michele Vecchione, Eldad Davidov, Ronald Fischer, Constanze Beierlein, Alice Ramos, Markku Verkasalo, Jan-Erik Lönnqvist, Kursad Demirutku, et al. 2012. Refining the theory of basic individual values. Journal of personality and social psychology, 103(4):663. find
  81. Haocong Rao, Cyril Leung, and Chunyan Miao. 2023. Can chatgpt assess human personalities? a general evaluation framework. arXiv preprint arXiv:2303.01248. find
  82. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. 2023. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548.
  83. R. Rezsohazy. 2001. Values, sociology of. In Neil J. Smelser and Paul B. Baltes, editors, International Encyclopedia of the Social Behavioral Sciences, pages 16153–16158. Pergamon, Oxford. find
  84. Gabriel Simmons. 2023. Moral mimicry: Large language models produce moral rationalizations tailored to political identity. find
  85. John P Robinson, Phillip R Shaver, and Lawrence S Wrightsman. 2013. Measures of personality and social psychological attitudes: Measures of social psychological attitudes, volume 1. Academic Press. find
  86. Leonard Simms, Lewis Goldberg, John Roberts, David Watson, John Welte, and Jane Rotterman. 2011. Computerized adaptive assessment of personality disorder: Introducing the cat-pd project. Journal of personality assessment, 93:380–9. find
  87. Milton Rokeach. 1974. The nature of human values. find
  88. Joanna Różycka-Tran, Paweł Boski, and Bogdan Wojciszke. 2015. Belief in a zero-sum game as a social axiom. Journal of Cross-Cultural Psychology, 46:525 – 548. find
  89. Bruce Smith, Jeanne Dalen, Kathryn Wiggins, Erin Tooley, Paulette Christopher, and Jennifer Bernard. 2008. The brief resilience scale: Assessing the ability to bounce back. International journal of behavioral medicine, 15:194–200. find
  90. Xiaoyang Song, Akshat Gupta, Kiyan Mohebbizadeh, Shujie Hu, and Anant Singh. 2023. Have large language models developed a personality?: Applicability of self-assessment tests in measuring personality in llms. arXiv preprint arXiv:2305.14693. — [arXiv:2305.14693](https://arxiv.org/abs/2305.14693) find
  91. Taylor Sorensen, Liwei Jiang, Jena D Hwang, Sydney Levine, Valentina Pyatkin, Peter West, Nouha Dziri, Ximing Lu, Kavel Rao, Chandra Bhagavatula, et al. 2024. Value kaleidoscope: Engaging ai with pluralistic human values, rights, and duties. In Proceedings of the AAAI Conference on Artificial Intell…
  92. Annette L. Stanton, Sarah B. Kirk, Christine L. Cameron, and Sharon Danoff-Burg. 2000. Coping through emotional approach: scale construction and validation. Journal of personality and social psychology, 78 6:1150–69. find
  93. Zhen Tan, Alimohammad Beigi, Song Wang, Ruocheng Guo, Amrita Bhattacharjee, Bohan Jiang, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. Large language models for data annotation: A survey. arXiv preprint arXiv:2402.13446. — [arXiv:2402.13446](https://arxiv.org/abs/2402.13446) find
  94. Auke Tellegen and Niels G Waller. 2008. Exploring personality through test construction: Development of the multidimensional personality questionnaire. The SAGE handbook of personality theory and assessment, 2:261–292. find
  95. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  96. Jeanne L. Tsai, Felicity F. Miao, Emma Seppala, Helene H Fung, and Dannii Yuen lan Yeung. 2007. Influence and adjustment goals: sources of cultural differences in ideal affect. Journal of personality and social psychology, 92 6:1102–17. find
  97. Jen tse Huang, Wenxuan Wang, Man Ho Lam, Eric John Li, Wenxiang Jiao, and Michael R. Lyu. 2023. Revisiting the reliability of psychological scales on large language models. find
  98. Xintao Wang, Quan Tu, Yaying Fei, Ziang Leng, and Cheng Li. 2023a. Does role-playing chatbots capture the character personalities? assessing personality traits for role-playing chatbots. find
  99. Zekun Moore Wang, Zhongyuan Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Man Zhang, et al. 2023b. Rolellm: Benchmarking, eliciting, and enhancing role-playing abilities of large language models. arXiv preprint arXiv:2310.00746.
  100. Zhilin Wang, Yu Ying Chiu, and Yu Cheung Chiu. 2023c. Humanoid agents: Platform for simulating human-like generative agents. arXiv preprint arXiv:2310.05418. find
  101. Yaning Xie. 1998. A preliminary study of the reliability and validity of the simplified coping strategies questionnaire. Chinese Journal of Clinical Psychology. find
  102. Jing Yao, Xiaoyuan Yi, Xiting Wang, Yifan Gong, and Xing Xie. 2023. Value fulcra: Mapping large language models to the multidimensional spectrum of basic human values. arXiv preprint arXiv:2311.10766. — [arXiv:2311.10766](https://arxiv.org/abs/2311.10766) find
  103. Haoran Ye, Jiarui Wang, Zhiguang Cao, and Guojie Song. 2024. Reevo: Large language models as hyper-heuristics with reflective evolution. find
  104. Ruoyu Zhang, Yanzeng Li, Yongliang Ma, Ming Zhou, and Lei Zou. 2023a. LLMaAA: Making large language models as active annotators. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13088–13103, Singapore. Association for Computational Linguistics. find
  105. Xin-an Zhang, Qing Cao, and Nicholas Grigoriou. 2011. Consciousness of social face: The development and validation of a scale measuring desire to gain face versus fear of losing face. The Journal of Social Psychology, 151:129 – 149. find
  106. Zhaowei Zhang, Fengshuo Bai, Jun Gao, and Yaodong Yang. 2023b. Measuring value understanding in language models through discriminator-critique gap. find
  107. Zhaowei Zhang, Nian Liu, Siyuan Qi, Ceyao Zhang, Ziqi Rong, Yaodong Yang, and Shuguang Cui. 2023c. Heterogeneous value evaluation for large language models. arXiv preprint arXiv:2305.17147. — [arXiv:2305.17147](https://arxiv.org/abs/2305.17147) find
  108. Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. 2023. Can large language models transform computational social science? arXiv preprint arXiv:2305.03514.
  109. William W.K. Zung. 1971. A rating instrument for anxiety disorders. Psychosomatics, 12(6):371–379. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ren-2024-valuebench-comprehensively-evaluating-value-orientations/