Elamur Research
☰
home / papers / 2023 / Personality Traits in Large Language Models

Personality Traits in Large Language Models

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2307.00184

Summary

Chatbots built on LLMs often appear to show a personality, but earlier attempts to measure it used questionnaires without checking whether the scores were statistically sound. This paper applies human psychometric standards to that question. It asks whether LLM-synthesized personality can be measured reliably and validly, and whether it can be deliberately steered.

The authors built a structured prompting scheme. Each test item is wrapped in varied persona descriptions, instructions and response scales, so every test is given to a model many times. These paired scores are then used to assess four properties: the consistency of the measurements, their convergent and discriminant validity, and how well they relate to external criteria. A second prompting method shapes Big Five traits at graded intensities using trait adjectives.

Across 18 models, the measurements were trustworthy only for some models and prompt setups. The evidence was strongest for larger, instruction-tuned models. Personality could be pushed toward target profiles, and the shaped traits carried over into ordinary generation tasks. This gives responsible-AI work a principled way to audit and control the persona a model presents.

Key points

  • Instruments: two Big Five inventories from different theoretical traditions, IPIP-NEO and the 44-item Big Five Inventory (BFI), plus 11 further tests of personality-related constructs used for criterion validity.

  • Prompt variation: 50 Biographic Descriptions were sampled from PersonaChat, a crowd-sourced set of 1,155 fictional profiles. Responses were resampled 1,250 times across combinations of descriptions and item instructions.

  • Metrics:

    • reliability via Cronbach's α and Guttman's λ6;
    • convergent validity via Pearson's r between IPIP-NEO and BFI domain scores;
    • criterion validity via correlations with non-personality measures.

    All three improved with model size and instruction tuning.

  • Shaping: traits were steered at nine levels with 104 trait adjectives. The effect was verified downstream: the Apply Magic Sauce (AMS) API scored the personality expressed in generated social media posts.

  • Release: the models tested included families such as PaLM, Llama 2 and Mistral 7B Instruct. Test scores, generated texts and code are public in a Google Cloud bucket and at github.com/google-deepmind/personality_in_llms.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing · AI & machine learning

Cited by

117 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 113 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2307.00184 · arXiv 2307.00184

@misc{serapiogarcia2023personality,
  title = {{Personality Traits in Large Language Models}},
  author = {Serapio-Garcia, Greg and Safdari, Mustafa and Crepy, Clement and Sun, Luning and Fitz, Stephen and Romero, Peter and Abdulhai, Marwa and Faust, Aleksandra and Mataric, Maja},
  year = {2023},
  doi = {10.48550/arxiv.2307.00184},
  eprint = {2307.00184},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2307.00184}
}

References

127 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. [1] Marwa Abdulhai, Clément Crepy, Daria Valter, John Canny, and Natasha Jaques. Moral foundations of large language models. In AAAI 2023 Workshop on Representation Learning for Responsible Human-Centric AI, 2022. find
  2. [9] B Ann Bettencourt and Cyndi Kernahan. A meta-analysis of aggression in the presence of violent cues: Effects of gender differences and aversive provocation. Aggressive Behavior, 23(6):447–456, 1997. find
  3. [2] G.W. Allport. Personality: A Psychological Interpretation. H. Holt, 1937. find
  4. [10] Wiebke Bleidorn, Patrick L Hill, Mitja D Back, Jaap JA Denissen, Marie Hennecke, Christopher J Hopwood, Markus Jokela, Christian Kandler, Richard E Lucas, Maike Luhmann, et al. The policy relevance of personality traits. American Psychologist, 74(9):1056, 2019. find
  5. [3] American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, editors. Standards for educational and psychological testing. American Educational Research Association, Lanham, MD, March 2014. find
  6. [11] Ryan L Boyd and James W Pennebaker. Language-based personality: A new approach to personality in a digital world. Current Opinion in Behavioral Sciences, [4] American Educational Research Association, American Psychological Association, National Council on Measurement in Vedant Misra, Evan Mori… find our card
  7. [12] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter,… find
  8. [16] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran… find
  9. [13] Donald T Campbell and Donald W Fiske. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2):81, 1959. find
  10. [14] Graham Caron and Shashank Srivastava. Identifying and manipulating the personality traits of language models. CoRR, abs/2212.10276, 2022. — [arXiv:2212.10276](https://arxiv.org/abs/2212.10276) find
  11. [15] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, M… find
  12. [27] James D Evans. Straightforward Statistics for the Behavioral Sciences. Brooks/Cole Publishing Co, 1996. find
  13. [18] Lee Anna Clark and David Watson. Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3):309, 1995. find
  14. [28] Ronald Fischer and Diana Boer. Motivational basis of personality traits: A meta-analysis of value-personality correlations. Journal of Personality, 83(5):491–510, 2015. find
  15. [19] Lee Anna Clark and David Watson. Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12):1412, 2019. find
  16. [29] Clémentine Fourrier, Nathan Habib, Alina Lozovskaya, Konrad Szafer, and Thomas Wolf. Open llm leaderboard v2. https://huggingface. co/spaces/open-llm-leaderboard/ open llm leaderboard, 2024. find
  17. [20] Paul T Costa, Jr. and Robert R McCrae. Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI): Professional Manual. Psychological Assessment Resources, Odessa, FL, 1992. find
  18. [30] Iason Gabriel. Artificial intelligence, values, and alignment. Minds and machines, 30(3):411–437, 2020. find
  19. [31] Iason Gabriel and Vafa Ghazavi. The Challenge of Value Alignment: From Fairer Algorithms to AI Safety. In The Oxford Handbook of Digital Ethics. Oxford University Press. [21] Charles B Crawford and George A Ferguson. A general rotation criterion and its use in orthogonal rotation. Psychometrika… find
  20. [32] Francis Galton. Measurement of character. Fortnightly Review, 36:179–85, 1884. find
  21. [22] Lee J Cronbach. Coefficient alpha and the internal structure of tests. Psychometrika, 16(3):297–334, 1951. find
  22. [33] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling. CoRR, abs/2101.00027, 2020. find
  23. [23] S. Crouse, G. Elbaz, and C. Malamud. Common Crawl Foundation., 2008. find
  24. [24] Colin G. DeYoung. Toward a theory of the Big Five. Psychological Inquiry, 21(1):26– 33, 2010. find
  25. [34] Lewis R Goldberg. Language and individual differences: The search for universals in personality lexicons. Review of Personality and Social Psychology, 2(1):141–165, 1981. find
  26. [25] Colin G DeYoung, Roger E Beaty, Erhan Genç, Robert D Latzman, Luca Passamonti, Michelle N Servaas, Alexander J Shack-man, Luke D Smillie, R Nathan Spreng, Essi Viding, et al. Personality neuroscience: An emerging field with bright prospects. Personality Science, 3:1–21, 2022. find
  27. [35] Lewis R Goldberg. The development of markers for the Big-Five factor structure. Psychological Assessment, 4(1):26–42, 1992. find
  28. [26] Colin G DeYoung, Jacob B Hirsh, Matthew S Shane, Xenophon Papademetris, Nallakkandi Rajeevan, and Jeremy R Gray. Testing predictions from personality neuroscience: Brain structure and the Big Five. Psychological Science, 21(6):820–828, 2010. find
  29. [36] Lewis R. Goldberg. A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several Five-Factor models. Personality Psychology in Europe, 7(1):7–28, 1999. find
  30. [45] Joel Jang, Seonghyeon Ye, and Minjoon Seo. Can large language models truly understand prompts? a case study with negated prompts. In Alon Albalak, Chunting Zhou, Colin Raffel, Deepak Ramachandran, Sebastian Ruder, and Xuezhe Ma, editors, Proceedings of The 1st Transfer Learning for Natural Lang… find
  31. [37] Alan K Goodboy and Matthew M Martin. Omega over alpha for reliability estimation of unidimensional communication measures. Annals of the International Communication Association, 44(4):422–439, 2020. find
  32. [38] Louis Guttman. A basis for analyzing test-retest reliability. Psychometrika, 10(4):255– 282, 1945. find
  33. [39] Thilo Hagendorff. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. CoRR, abs/2303.13988, 2023. — [arXiv:2303.13988](https://arxiv.org/abs/2303.13988) find
  34. [46] Kristin Jankowsky, Gabriel Olaru, and Ulrich Schroeders. Compiling measurement invariant short scales in cross-cultural personality assessment using ant colony optimization. European Journal of Personality, 34(3):470–485, 2020. find
  35. [40] Christopher Hare and Keith T. Poole. Psychometric Methods in Political Science, chapter 28, pages 901–931. John Wiley & Sons, Ltd, 2018. find
  36. [47] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée L… find
  37. [41] Steven J. Heine and Emma E. Buchtel. Personality: The universal and the culturally specific. Annual Review of Psychology, 60(1):369–394, 2009. find
  38. [42] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021. find our card
  39. [48] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre… find
  40. [43] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Si… find
  41. [49] Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. Evaluating and inducing personality in pre-trained language models. CoRR, abs/2206.07550, 2023. find
  42. [44] Abigail Z. Jacobs. Measurement as governance in and for responsible AI. CoRR, abs/2109.05658, 2021. — [arXiv:2109.05658](https://arxiv.org/abs/2109.05658) find
  43. [50] Hang Jiang, Xiajie Zhang, Xubo Cao, and Jad Kabbara. Personallm: Investigating [58] Ahmet Baki Kocaballi, Shlomo Berkovsky, Juan C Quiroz, Liliana Laranjo, Huong Ly Tong, Dana Rezazadegan, Agustina Briatore, and Enrico Coiera. The personalization of conversational agents in health care: Systema… find
  44. [51] Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962–977, 09 2021. find
  45. [59] Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 22199–22213. Cur… find our card
  46. [52] Oliver P. John, Laura P. Naumann, and Christopher J. Soto. Paradigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issues. In Oliver P. John, Richard W. Robbins, and Lawrence A. Pervin, editors, Handbook of Personality: Theory and Research, pages 114–158.… find
  47. [60] Michal Kosinski, Sandra C Matz, Samuel D Gosling, Vesselin Popov, and David Stillwell. Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist, 70(6):543, 2015. find
  48. [53] Oliver P. John and Sanjay Srivastava. The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In L. A. Pervin and Oliver P. John, editors, Handbook of Personality: Theory and Research, volume 2, pages 102–138. Guilford Press, New York, 1999. find
  49. [61] Michal Kosinski, David Stillwell, and Thore Graepel. Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110(15):5802–5805, 2013. find
  50. [54] Henry F Kaiser and John Rice. Little Jiffy, Mark IV. Educational and Psychological Measurement, 34(1):111–117, 1974. find
  51. [55] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, abs/2001.08361, 2020. find
  52. [62] Roman Kotov, Wakiza Gamez, Frank Schmidt, and David Watson. Linking “big” personality traits to anxiety, depressive, and substance use disorders: A meta-analysis. Psychological Bulletin, 136(5):768, 2010. find
  53. [63] Jon A Krosnick and Duane F Alwin. An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly, 51(2):201–219, 1987. find
  54. [56] Saketh Reddy Karra, Son The Nguyen, and Theja Tulabandhula. Estimating the personality of white-box language models. CoRR, abs/2204.12000, 2023. — [arXiv:2204.12000](https://arxiv.org/abs/2204.12000) find
  55. [64] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating [57] Maciej Karwowski, Iz… find
  56. Systems Principles, 2023. find
  57. [73] Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330, 1993. find our card
  58. [65] Kibeom Lee and Michael C. Ashton. Psychometric properties of the HEXACO Personality Inventory. Multivariate Behavioral Research, 39(2):329–358, 2004. find
  59. [74] Sandra Matz, Michal Kosinski, David Stillwell, and Gideon Nave. Psychological framing as an effective approach to real-life persuasive communication. ACR North American Advances, 2017. find
  60. [66] Xingxuan Li, Yutong Li, Shafiq Joty, Linlin Liu, Fei Huang, Lin Qiu, and Lidong Bing. Does GPT-3 demonstrate psychopathy? evaluating large language models from a psychological perspective. CoRR, abs/2212.10529, 2023. — [arXiv:2212.10529](https://arxiv.org/abs/2212.10529) find
  61. [75] Robert R McCrae and Antonio Terracciano. Universal features of personality traits from the observer’s perspective: Data from 50 cultures. Journal of Personality and Social Psychology, 88(3):547, 2005. find
  62. [67] Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. Towards understanding and mitigating social biases in language models. CoRR, abs/2106.13219, 2021. find our card
  63. [76] Roderick P McDonald. Test theory: A unified treatment. Lawrence Erlbaum Associates Publishers, 1999. find
  64. [68] Rensis Likert. A Technique for the Measurement of Attitudes. Number 136– 165. Archives of Psychology, 1932. find
  65. [77] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2017. find our card
  66. [69] Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3,214–3,252, Dublin, Ireland, May 2022. Association for Computational… find our card
  67. [78] Samuel Messick. Standards of validity and the validity of standards in performance asessment. Educational Measurement: Issues and Practice, 14(4):5–8, 1995. find
  68. [70] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. CoRR, abs/2307.03172, 2023. find our card
  69. [79] Samuel Messick. Test validity: A matter of consequence. Social Indicators Research, 45:35–44, 1998. find
  70. [80] Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? CoRR, abs/2202.12837, 2022. find our card
  71. [71] Rabeeh Karimi Mahabadi, Luke Zettlemoyer, James Henderson, Marzieh Saeidi, Lambert Mathias, Veselin Stoyanov, and Majid Yazdani. Perfect: Prompt-free and efficient few-shot learning with language models. CoRR, abs/2204.01172, 2022. find
  72. [81] Marilù Miotto, Nicola Rossberg, and Bennett Kleinberg. Who is GPT-3? an exploration of personality, values and demographics. In Proceedings of the Fifth Workshop on Natural Language Processing and Computational Social Science (NLP+CSS), pages 218–227, Abu Dhabi, UAE, November 2022. Association… find
  73. Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding, Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine McLeavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czarnecki, Colin Jarvis, Colin… find
  74. [83] Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi. Auditing large language models: A three-layered approach. AI and Ethics, pages 1–31, 2023. find
  75. [84] Daniel Nettle. The evolution of personality variation in humans and other animals. American Psychologist, 61(6):622, 2006. find
  76. [85] University of rics Centre. API. Cambridge Psychomet-Apply Magic Sauce [86] OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Madry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Ale… find our card
  77. [88] OpenAI. GPT-4 technical report. CoRR, abs/2303.08774, 2023. find our card
  78. [89] OpenAI. GPT-4o mini: advancing cost-efficient intelligence, 2024. find
  79. [90] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Traini… find
  80. [91] Keyu Pan and Yawen Zeng. Do llms possess a personality? making the mbti test an amazing evaluation for large language [99] Guanghui Qin, Yukun Feng, and Benjamin Van Durme. The NLP task effectiveness of long-range transformers. In Proceedings of the 17th Conference of the European Chapter of th… find
  81. [92] Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Proceedings of the 54th Annual Meeting of the Association f… find our card
  82. [100] Boele De Raad, Marco Perugini, Martina Hrebı́cková, and Piotr Szarota. Lingua franca of personality: Taxonomies and structures based on the psycholexi-cal approach. Journal of Cross-Cultural Psychology, 29(1):212–232, 1998. find
  83. [93] Gregory Park, H Andrew Schwartz, Johannes C Eichstaedt, Margaret L Kern, Michal Kosinski, David J Stillwell, Lyle H Ungar, and Martin EP Seligman. Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108(6):934, 2015. find
  84. [101] Brent W. Roberts. A revised sociogenomic model of personality traits. Journal of Personality, 86(1):23–35, 2018. find
  85. [102] Brent W. Roberts, Nathan R. Kuncel, Rebecca Shiner, Avshalom Caspi, and Lewis R. Goldberg. The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological science, 2(4):3… find
  86. [94] H. Y. Park, B. M. Wiernik, I. Oh, E. Gonzalez-Mulé, D. S. Ones, and Y. Lee. Meta-analytic five-factor model personality intercorrelations: Eeny, meeny, miney, moe, how, which, why, and where to go. Journal of Applied Psychology, 105:1490–1529, 2020. find
  87. [95] Laura Parks-Leduc, Gilad Feldman, and Anat Bardi. Personality traits and personal values: A meta-analysis. Personality and Social Psychology Review, 19(1):3–29, 2015. find
  88. [103] Brent W. Roberts and Hee J. Yoon. Personality psychology. Annual Review of Psychology, 73(1):489–516, 2022. find
  89. [104] Jean-Pierre Rolland. The cross-cultural generalizability of the Five-Factor model of personality. In Robert R. McCrae and Jüri Allik, editors, The Five-Factor Model of Personality Across Cultures, pages 7–28. Springer US, Boston, MA, 2002. find
  90. [96] Max Pellert, Clemens M Lechner, Claudia Wagner, Beatrice Rammstedt, and Markus Strohmaier. Large language models open up new opportunities and challenges for psychometric assessment of artificial intelligence. October 2022. find
  91. [105] John Rust, Michal Kosinski, and David Stillwell. Modern Psychometrics: The Science of Psychological Assessment. Routledge, 4 edition, 2020. find
  92. [97] James W Pennebaker and Laura A King. Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology, 77(6):1296, 1999. find
  93. [106] Gerard Saucier and Lewis R Goldberg. Lexical studies of indigenous personality factors: Premises, products, and prospects. Journal of Personality, 69(6):847–879, 2001. find
  94. [98] Philip M Podsakoff, Scott B MacKenzie, Jeong-Yeon Lee, and Nathan P Podsakoff. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5):879–903, 2003. find
  95. [107] Patrick Schramowski, Cigdem Turan, Nico Andersen, Constantin A Rothkopf, and [115] Randy Stein and Alexander B. Swan. Evaluating the validity of Myers-Briggs Type Indicator theory: A teaching tool and window into intuitive psychology. Social and Personality Psychology Compass, 13(2):e12434, 20… find
  96. Kristian Kersting. Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence, 4(3):258–268, 2022. find
  97. [108] H. Andrew Schwartz, Johannes C. Eichstaedt, Margaret L. Kern, Lukasz Dziurzynski, Stephanie M. Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin E. P. Seligman, and Lyle H. Ungar. Personality, gender, and age in the language of social media: The open-vocabulary appro… find
  98. [116] Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, and Mohit Iyyer. Do long-range language models actually use long-range context? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 807– 822, Online and Punta Cana, Dominican Republic, November 2… find
  99. [109] Gregory Serapio-Garcı́a, Dasha Valter, and Clément Crepy. PsyBORGS: Psychometric Benchmark of Racism, Generalization, and Stereotyping. [117] Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. The science of detecting LLM-generated texts. CoRR, abs/2303.07205, 2023. — [arXiv:2303.07205](https://arxiv.org/abs/2303.07205) find
  100. [110] Amy Shaw, Melissa Kapnek, and Neil A Morelli. Measuring creative self-efficacy: An item response theory analysis of the Creative Self-Efficacy Scale. Frontiers in Psychology, 12:678033, 2021. find
  101. [118] Adriana Tapus, Cristian Ţăpuş, and Maja J Matarić. User–robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy. Intell. Serv. Robot., 1(2):169–183, April 2008. find
  102. [111] Kurt Shuster, Mojtaba Komeili, Leonard Adolphs, Stephen Roller, Arthur Szlam, and Jason Weston. Language models that seek for knowledge: Modular search & generation for dialogue and prompt completion. CoRR, abs/2203.13224, 2022. find
  103. [119] Mikke Tavast, Anton Kunnari, and Perttu Hämäläinen. Language models can generate human-like self-reports of emotion. pages 69–72, 2022. find
  104. [120] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR,… find
  105. [112] Leonard Simms, Trevor F. Williams, and Ericka Nus Simms. Assessment of the Five Factor Model. In Thomas A. Widiger, editor, The Oxford Handbook of the Five Factor Model, pages 353–380. Oxford University Press, 05 2017. find
  106. [113] Umarpreet Singh and Parham Aarabhi. Can AI have a personality? In 2023 IEEE Conference on Artificial Intelligence (CAI), pages 205–206, 2023. find
  107. [121] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian F… — [arXiv:2305.14693](https://arxiv.org/abs/2305.14693) find
  108. Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Re… find
  109. Journal of Personality, 60(2):441–476, 1992. find
  110. [127] David Wechsler. The measurement of adult intelligence (3rd ed.). Williams & Wilkins Co, Baltimore, 1946. find
  111. [128] Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022. find
  112. [122] Tomer Ullman. Large language models fail on trivial alterations to theory-of-mind tasks. CoRR, abs/2302.08399, 2023. find
  113. [129] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Transactions on Ma… find our card
  114. [123] Ertugrul Uysal, Sascha Alavi, and Valéry Bezençon. Trojan horse or useful helper? a relationship perspective on artificial intelligence assistants with humanlike features. Journal of the Academy of Marketing Science, pages 1–23, 2022. find
  115. [130] Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma. Larger language models do in-context learning differently. CoRR, abs/2303.03846, 2023. find our card
  116. [124] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proce… find
  117. [131] Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haa… find
  118. [125] Xintao Wang, Yunze Xiao, Jen tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. Incharacter: Evaluating personality fidelity in role-playing agents through psychological interviews. 2024. find
  119. [132] Brandon T Willard and Rémi Louf. Efficient guided generation for llms. arXiv preprint arXiv:2307.09702, 2023. find
  120. [126] David Watson and Lee Anna Clark. On traits and temperament: General and specific factors of emotional experience and their relation to the Five-Factor model. [133] Zhewei Yao, Cheng Li, Xiaoxia Wu, Stephen Youn, and Yuxiong He. A comprehensive study on post-training quantization for large [140… — [arXiv:2303.08302](https://arxiv.org/abs/2303.08302) find
  121. [134] J Kenneth Young, Beaujean, and A Alexander. Measuring personality in wave I of the national longitudinal study of adolescent health. Front. Psychol., 2:158, July 2011. find
  122. [141] Richard E Zinbarg, William Revelle, Iftah Yovel, and Wen Li. Cronbach’s α, revelle’s β, and mcdonald’s ω h: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70:123–133, 2005. find
  123. [135] Wu Youyou, Michal Kosinski, and David Stillwell. Computer-based personality judgments are more accurate than those made by humans. Proceedings of the National Academy of Sciences, 112(4):1036–1040, 2015. find
  124. [136] J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. Why Johnny can’t prompt: How non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, New York, NY, USA, 2023. Association for Comput… find
  125. [137] Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbou… find our card
  126. [138] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of l… find our card
  127. [139] Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. CoRR, abs/1909.08593, 2020. find