Elamur Research⌕
☰
home / papers / 2025 / BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data

BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data

Paper · 2025 · Wenkai Li, Jiarui Liu, Andy Liu et al.

Summary

Earlier attempts to give LLMs a Big Five personality mostly rely on prompts that describe typical behaviours. These descriptions are often borrowed from the same questionnaires later used for evaluation, which raises validity concerns. They also make little sense for a text-only model. The authors instead ground personality in how real people write. Their PSYCHSTEER pipeline trains trait-specific expert generators on PsychGenerator, a collection of Facebook posts labelled with Big Five scores. These experts steer a base model through DExperts decoding, while SODA supplies the social scenarios. The result is BIG5-CHAT, a corpus of 100,000 dialogues covering high and low levels of each trait. Models trained on it with SFT and DPO show stronger target traits on BFI and IPIP-NEO than instruction- or demonstration-prompted models. Their between-trait correlations are also closer to human data. On reasoning benchmarks, the induced traits shift performance in directions that echo findings from human psychology. The work offers a data-driven and more valid route to simulating personas in dialogue systems.

Key points

  • BIG5-CHAT dataset: 100,000 personality-conditioned dialogues, built by combining PsychGenerator (about 850K annotated Facebook posts) with SODA scenarios via DExperts. Data and code are released at github.com/jiarui-liu/Big5Chat.
  • Methods compared: SFT and DPO versus prompting. Both training methods outperform prompting on the BFI and IPIP-NEO tests and show more realistic intra-trait correlations.
  • Reasoning benchmarks: SocialIQA, GSM8K, MathQA, TruthfulQA, CommonsenseQA, PIQA, MMLU and GPQA.
  • Trait effects on reasoning: Models trained toward higher conscientiousness and agreeableness, or lower extraversion and neuroticism, generally reason better.
  • Limitations: The study steers only one trait at a time. PsychGenerator skews young (median age 22) and toward Facebook users, which raises fairness concerns. The authors also flag misuse risks such as impersonation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksClinical Psychology · 20 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona T. Diab and Maarten Sap. 2025. BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.999

@inproceedings{li2025big5,
  title = {{BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Data}},
  author = {Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona T. Diab and Maarten Sap},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.999},
  url = {https://doi.org/10.18653/v1/2025.acl-long.999}
}

References

127 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Phillip L Ackerman and Eric D Heggestad. 1997. Intelligence, personality, and interests: evidence for overlapping traits. Psychological bulletin, 121(2):219. find
  2. Rangina Ahmad, Dominik Siemon, Ulrich Gnewuch, and Susanne Robra-Bissantz. 2022. Designing personality-adaptive conversational agents for mental health care. Information Systems Frontiers, 24(3):923–943. find
  3. Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. arXiv preprint arXiv:1905.13319.
  4. Anthropic. 2024. Claude’s character. Accessed: 2024-08-30. find
  5. Michael C Ashton, Kibeom Lee, and Sampo V Paunonen. 2002. What is the central feature of extraversion? social attention versus reward sensitivity. Journal of personality and social psychology, 83(1):245. find
  6. Mark W Baldwin. 1992. Relational schemas and the processing of social information. Psychological bulletin, 112(3):461. find
  7. Albert Bandura, Dorothea Ross, and Sheila A Ross. 1961. Transmission of aggression through imitation of aggressive models. The Journal of Abnormal and Social Psychology, 63(3):575. find
  8. Murray R Barrick and Michael K Mount. 1991. The big five personality dimensions and job performance: a meta-analysis. Personnel psychology, 44(1):1–26. find
  9. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439.
  10. Gerhard Blickle. 1996. Personality traits, learning stratigies, and performance. European Journal of personality, 10(5):337–352. find
  11. Raymond B Cattell. 1957. Personality and motivation structure and measurement. find
  12. Tomas Chamorro-Premuzic and Adrian Furnham. 2006. Intellectual competence and the intelligent personality: A third way in differential psychology. Review of General Psychology, 10(3):251–267. find
  13. Tomas Chamorro-Premuzic, Adrian Furnham, and Konstantinos Petrides. 2006. Personality and intelligence. Journal of Individual Differences, 27(3):147–150. find
  14. Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Xing Gao, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, Fei Huang, et al. 2024a. Socialbench: Sociality evaluation of role-playing conversational agents. arXiv preprint arXiv:2403.13679. find
  15. Nuo Chen, Y Wang, Yang Deng, and Jia Li. 2024b. The oscars of ai theater: A survey on role-playing with language models. arXiv preprint arXiv:2407.11484. — [arXiv:2407.11484](https://arxiv.org/abs/2407.11484) find
  16. Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. 2022. Large language models meet harry potter: A bilingual dataset for aligning dialogue agents with characters. arXiv preprint arXiv:2211.06869. — [arXiv:2211.06869](https://arxiv.org/abs/2211.06869) find
  17. Myra Cheng, Tiziano Piccardi, and Diyi Yang. 2023. Compost: Characterizing and evaluating caricature in llm simulations. arXiv preprint arXiv:2310.11501. find
  18. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  19. Paul T Costa and Robert R McCrae. 2008. The revised neo personality inventory (neo-pi-r). The SAGE handbook of personality theory and assessment, 2(2):179– 198. find
  20. Paul T Costa Jr, James L Fozard, Robert R McCrae, and Raymond Bossé. 1976. Relations of age and personality dimensions to cognitive ability factors. Journal of gerontology, 31(6):663–669. find
  21. Jiaxi Cui, Liuzhenghao Lv, Jing Wen, Jing Tang, YongHong Tian, and Li Yuan. 2023. Machine mindset: An mbti exploration of large language models. arXiv preprint arXiv:2312.12999. — [arXiv:2312.12999](https://arxiv.org/abs/2312.12999) find
  22. Jia Deng, Tianyi Tang, Yanbin Yin, Wenhao Yang, Wayne Xin Zhao, and Ji-Rong Wen. 2024. Neuron-based personality trait induction in large language models. Preprint, arXiv:2410.12327. — [arXiv:2410.12327](https://arxiv.org/abs/2410.12327) find
  23. John M Digman. 1990. Personality structure: Emergence of the five-factor model. Annual review of psychology, 41(1):417–440. find
  24. Wenhan Dong, Yuemeng Zhao, Zhen Sun, Yule Liu, Zifan Peng, Jingyi Zheng, Zongmin Zhang, Ziyi Zhang, Jun Wu, Ruiming Wang, et al. 2025. Humanizing llms: A survey of psychological measurements with tools, datasets, and human-agent applications. arXiv preprint arXiv:2505.00049. — [arXiv:2505.00049](https://arxiv.org/abs/2505.00049) find
  25. Florian E Dorner, Tom Sühr, Samira Samadi, and Augustin Kelava. 2023. Do personality tests generalize to large language models? arXiv preprint arXiv:2311.05297. — [arXiv:2311.05297](https://arxiv.org/abs/2311.05297) find
  26. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez,… — The Llama 3 Herd of Models
  27. L Edwards. 2022. The eu ai act: a summary of its significance and scope, ada lovelace institute. find
  28. Jon Elster. 2015. Explaining social behavior: More nuts and bolts for the social sciences. Cambridge University Press. find
  29. HJ Eysenck. 1997. Eysenck personality questionnaire-revised (epq-r) and short scale (epq-rs). Madrid: TEA Ediciones. find
  30. Michael W Eysenck. 2013. Anxiety: The cognitive perspective. Psychology Press. find
  31. Iustin Floroiu. 2024. Big5personalityessays: Introducing a novel synthetic generated dataset consisting of short state-of-consciousness essays annotated based on the five factor model of personality. arXiv preprint arXiv:2407.17586. — [arXiv:2407.17586](https://arxiv.org/abs/2407.17586) find
  32. Ivar Frisch and Mario Giulianelli. 2024. Llm agents in interaction: Measuring personality consistency and linguistic alignment in interacting populations of large language models. arXiv preprint arXiv:2402.02896. find
  33. Adrian Furnham. 1996. The big five versus the big four: the relationship between the myers-briggs type indicator (mbti) and neo-pi five factor model of personality. Personality and individual differences, 21(2):303– 307. find
  34. Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2024a. Large language models empowered agent-based modeling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1):1–24. find
  35. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… find
  36. Lewis R Goldberg. 1992. The development of markers for the big-five factor structure. Psychological assessment, 4(1):26. find
  37. Eileen K Graham and Margie E Lachman. 2012. Personality stability is associated with better cognitive performance in adulthood: are the stable more able? find
  38. Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 67(5):545–554. find
  39. WG Graziano. 1997. Agreeableness: A dimension of personality. find
  40. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseek-coder: When the large language model meets programming– the rise of code intelligence. arXiv preprint arXiv:2401.14196. find
  41. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  42. Dirk Hovy and Shannon L Spruit. 2016. The social impact of natural language processing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598.
  43. Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024b. Personallm: Investigating the ability of large language models to express personality traits. Preprint, arXiv:2305.02547. find
  44. Oliver P John, Laura P Naumann, and Christopher J Soto. 2008. Paradigm shift to the integrative big five trait taxonomy. Handbook of personality: Theory and research, 3(2):114–158. find
  45. Oliver P John, Sanjay Srivastava, et al. 1999. The big-five trait taxonomy: History, measurement, and theoretical perspectives. Handbook of personality: theory and research. find
  46. John A Johnson. 2014. Measuring thirty facets of the five factor model with a 120-item public domain inventory: Development of the ipip-neo-120. Journal of research in personality, 51:78–89. find
  47. Peter Karl Jonason and Gregory D. Webster. 2010. The dirty dozen: a concise measure of the dark triad. Psychological assessment, 22 2:420–32. find
  48. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
  49. Junko Kanero, Cansu Oranç, Sümeyye Koşkulu, G Tar-can Kumkale, Tilbe Göksun, and Aylin C Küntay. 2022. Are tutor robots for everyone? the influence of attitudes, anxiety, and personality on robot-led language learning. International Journal of Social Robotics, 14(2):297–312. find
  50. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  51. Saketh Reddy Karra, Son The Nguyen, and Theja Tulabandhula. 2022. Estimating the personality of white-box language models. arXiv preprint arXiv:2204.12000. — [arXiv:2204.12000](https://arxiv.org/abs/2204.12000) find
  52. Jen-tse Huang, Wenxuan Wang, Man Ho Lam, Eric John Li, Wenxiang Jiao, and Michael R. Lyu. 2023. Revisiting the reliability of psychological scales on large language models. arXiv preprint arXiv:2305.19926. — [arXiv:2305.19926](https://arxiv.org/abs/2305.19926) find
  53. Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, and Yejin Choi. 2023a. Soda: Million-scale dialogue distillation with social commonsense contextualization. Preprint, arXiv:2212.10465.
  54. Jen-tse Huang, Wenxuan Wang, Eric John Li, Man Ho Lam, Shujie Ren, Youliang Yuan, Wenxiang Jiao, Zhaopeng Tu, and Michael R. Lyu. 2024. On the humanity of conversational ai: Evaluating the psychological portrayal of llms. In Proceedings of the Twelfth International Conference on Learning Representat… find
  55. Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2023a. Evaluating and inducing personality in pre-trained language models. In NeurIPS. find
  56. Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2024a. Evaluating and inducing personality in pre-trained language models. Advances in Neural Information Processing Systems, 36. find
  57. Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2023b. Personallm: Investigating the ability of large language models to express personality traits. arXiv preprint arXiv:2305.02547. find
  58. Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, et al. 2022. Soda: Million-scale dialogue distillation with social commonsense contextualization. arXiv preprint arXiv:2212.10465.
  59. Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023b. FANToM: A benchmark for stress-testing machine theory of mind in interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14397–14413, Singapore.… find
  60. Hadas Kotek, Rikker Dockum, and David Sun. 2023. Gender bias and stereotypes in large language models. In Proceedings of the ACM collective intelligence conference, pages 12–24. find
  61. Nikola Kovačević, Christian Holz, Markus Gross, and Rafael Wampfler. 2024. The personality dimensions gpt-3 expresses during human-chatbot interactions. find
  62. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 8(2). find
  63. Seongyun Lee, Sue Hyun Park, Seungone Kim, and Minjoon Seo. 2024. Aligning to thousands of preferences via system message generalization. arXiv preprint arXiv:2405.17977. — [arXiv:2405.17977](https://arxiv.org/abs/2405.17977) find
  64. Jiale Li, Jiayang Li, Jiahao Chen, Yifan Li, Shijie Wang, Hugo Zhou, Minjun Ye, and Yunsheng Su. 2024. Evolving agents: Interactive simulation of dynamic and diverse human personalities. arXiv preprint arXiv:2404.02718. — [arXiv:2404.02718](https://arxiv.org/abs/2404.02718) find
  65. Xingxuan Li, Yutong Li, Shafiq Joty, Linlin Liu, Fei Huang, Lin Qiu, and Lidong Bing. 2022. Does gpt-3 demonstrate psychopathy? evaluating large language models from a psychological perspective. arXiv preprint arXiv:2212.10529. — [arXiv:2212.10529](https://arxiv.org/abs/2212.10529) find
  66. Q. Vera Liao and Jennifer Wortman Vaughan. 2024. AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap. Harvard Data Science Review, (Special Issue 5). Https://hdsr.mitpress.mit.edu/pub/aelql9qy. find
  67. Filip Lievens, Charlie L Reeve, and Eric D Heggestad. 2007. An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. Journal of Applied Psychology, 92(6):1672. find
  68. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958.
  69. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A Smith, and Yejin Choi. 2021. Dexperts: Decoding-time controlled text generation with experts and anti-experts. arXiv preprint arXiv:2105.03023.
  70. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692.
  71. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786.
  72. François Mairesse, Marilyn A Walker, Matthias R Mehl, and Roger K Moore. 2007. Using linguistic cues for the automatic recognition of personality in conversation and text. Journal of artificial intelligence research, 30:457–500.
  73. Shengyu Mao, Ningyu Zhang, Xiaohan Wang, Mengru Wang, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2023. Editing personality for llms. arXiv preprint arXiv:2310.02168. — [arXiv:2310.02168](https://arxiv.org/abs/2310.02168) find
  74. Gerald Matthews and Kirby Gilliland. 1999. The personality theories of hj eysenck and ja gray: A comparative review. Personality and Individual differences, 26(4):583–626. find
  75. Robert R McCrae. 1987. Creativity, divergent thinking, and openness to experience. Journal of personality and social psychology, 52(6):1258. find
  76. Robert R McCrae and Paul T Costa. 1987. Validation of the five-factor model of personality across instruments and observers. Journal of personality and social psychology, 52(1):81. find
  77. Robert R McCrae and Paul T Costa Jr. 1997. Personality trait structure as a human universal. American psychologist, 52(5):509. find
  78. Robert R McCrae and Oliver P John. 1992. An introduction to the five-factor model and its applications. Journal of personality, 60(2):175–215. find
  79. Matthias R Mehl, Samuel D Gosling, and James W Pennebaker. 2006. Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life. Journal of personality and social psychology, 90(5):862. find
  80. Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. 2024. State of what art? a call for multi-prompt llm evaluation. Transactions of the Association for Computational Linguistics, 12:933–949. find
  81. Joanna Moutafi, Adrian Furnham, and John Crump. 2003. Demographic and personality predictors of intelligence: A study using the neo personality inventory and the myers–briggs type indicator. European Journal of Personality, 17(1):79–94. find
  82. Alberto Muñoz-Ortiz, Carlos Gómez-Rodríguez, and David Vilares. 2023. Contrasting linguistic patterns in human and llm-generated text. arXiv preprint arXiv:2308.09067. — [arXiv:2308.09067](https://arxiv.org/abs/2308.09067) find
  83. Isabel Briggs Myers et al. 1962. The myers-briggs type indicator, volume 34. Consulting Psychologists Press Palo Alto, CA. find
  84. Carlos Olea, Holly Tucker, Jessica Phelan, Cameron Pattison, Shen Zhang, Maxwell Lieb, and J White. 2024. Evaluating persona prompting for question answering tasks. In Proceedings of th e 10th international conference on artificial intelligence and soft computing, Sydney, Australia. find
  85. OpenAI. 2024. Hello, gpt-4 turbo. Accessed: 2024-10-01. find
  86. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  87. Keyu Pan and Yawen Zeng. 2023. Do llms possess a personality? making the mbti test an amazing evaluation for large language models. arXiv preprint arXiv:2307.16180.
  88. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. find
  89. Delroy L Paulhus and Kevin M Williams. 2002. The dark triad of personality: Narcissism, machiavellianism, and psychopathy. Journal of research in personality, 36(6):556–563. find
  90. James W Pennebaker and Laura A King. 1999. Linguistic styles: language use as an individual difference. Journal of personality and social psychology, 77(6):1296. find
  91. Nikolay B Petrov, Gregory Serapio-García, and Jason Rentfrow. 2024. Limited ability of llms to simulate human psychological behaviours: a psychometric analysis. arXiv preprint arXiv:2405.07248. — [arXiv:2405.07248](https://arxiv.org/abs/2405.07248) find
  92. David J Pittenger. 1993. The utility of the myers-briggs type indicator. Review of educational research, 63(4):467–488. find
  93. Arthur E Poropat. 2009. A meta-analysis of the five-factor model of personality and academic performance. Psychological bulletin, 135(2):322. find
  94. Alisha Pradhan and Amanda Lazar. 2021. Hey google, do you have a personality? designing personality and personas for conversational agents. In Proceedings of the 3rd Conference on Conversational User Interfaces, pages 1–4. find
  95. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.
  96. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022.
  97. Brent W Roberts, Carl Lejuez, Robert F Krueger, Jessica M Richards, and Patrick L Hill. 2014. What is conscientiousness and how can it be assessed? Developmental psychology, 50(5):1315. find
  98. Michael D Robinson and Maya Tamir. 2005. Neuroticism as mental noise: a relation between neuroticism and reaction time standard deviations. Journal of personality and social psychology, 89(1):107. find
  99. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv:1904.09728.
  100. Toru Sato. 2005. The eysenck personality questionnaire brief version: Factor structure and reliability. The Journal of psychology, 139(6):545–552. find
  101. K Warner Schaie, Sherry L Willis, and Grace IL Caskie. 2004. The seattle longitudinal study: Relationship between personality and cognition. Aging Neuropsychology and Cognition, 11(2-3):304–324. find
  102. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2023. Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11324. find
  103. SM Seals and Valerie L Shalin. 2023. Long-form analogies generated by chatgpt lack human-like psycholinguistic properties. arXiv preprint arXiv:2306.04537. — [arXiv:2306.04537](https://arxiv.org/abs/2306.04537) find
  104. Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184.
  105. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2024. You don’t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. In Proceedings of t… find
  106. Stephen Soldz and George E Vaillant. 1999. The big five personality traits and the life course: A 45-year longitudinal study. Journal of research in personality, 33(2):208–232. find
  107. Nikita Soni, H Andrew Schwartz, João Sedoc, and Niranjan Balasubramanian. 2023. Large human language models: A need and the challenges. arXiv preprint arXiv:2312.07751. find
  108. Christopher J Soto, Oliver P John, Samuel D Gosling, and Jeff Potter. 2011. Age differences in personality traits from 10 to 65: Big five domains and facets in a large cross-sectional sample. Journal of personality and social psychology, 100(2):330. find
  109. Lee A Spitzley, Xinran Wang, Xunyu Chen, Judee K Burgoon, Norah E Dunbar, and Saiying Ge. 2022. Linguistic measures of personality in group discussions. Frontiers in Psychology, 13:887616. find
  110. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008– 3021. find
  111. Jose Sá, Andreas Kaltenbrunner, Jacopo Amidei, and Rubén Nieto. 2024. How well do simulated populations with gpt-4 align with real ones in clinical trials? the case of the epqr-a personality test. find
  112. Haoran Ye, Jing Jin, Yuhang Xie, Xin Zhang, and Guojie Song. 2025. Large language model psychometrics: A systematic review of evaluation, validation, and enhancement. arXiv preprint arXiv:2505.08245. — [arXiv:2505.08245](https://arxiv.org/abs/2505.08245) find
  113. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  114. Moshe Zeidner. 2005. Test anxiety: The state of the art. find
  115. Fiona Anting Tan, Gerard Christopher Yeo, Fanyou Wu, Weijie Xu, Vinija Jain, Aman Chadha, Kokil Jaidka, Yang Liu, and See-Kiong Ng. 2024. Phantom: Personality has an effect on theory-of-mind reasoning in large language models. arXiv preprint arXiv:2403.02246.
  116. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. find
  117. Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. 2024. CharacterEval: A Chinese benchmark for role-playing conversational agent evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1183…
  118. Zheni Zeng, Jiayi Chen, Huimin Chen, Yukun Yan, Yuxuan Chen, Zhenghao Liu, Zhiyuan Liu, and Maosong Sun. 2024a. Persllm: A personified training approach for large language models. Preprint, arXiv:2407.12393. find
  119. Zheni Zeng, Jiayi Chen, Huimin Chen, Yukun Yan, Yuxuan Chen, Zhiyuan Liu, and Maosong Sun. 2024b. Persllm: A personified training approach for large language models. arXiv preprint arXiv:2407.12393. find
  120. Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. 2024. SafetyBench: Evaluating the safety of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap…
  121. Mingqian Zheng, Jiaxin Pei, and David Jurgens. 2023. Is" a helpful assistant" the best role for large language models? a systematic evaluation of social roles in system prompts. arXiv preprint arXiv:2311.10054. — [arXiv:2311.10054](https://arxiv.org/abs/2311.10054) find
  122. Minjun Zhu, Linyi Yang, and Yue Zhang. 2024. Personality alignment of large language models. Preprint, arXiv:2408.11779.
  123. Lorainne Tudor Car, Dhakshenya Ardhithy Dhinagaran, Bhone Myint Kyaw, Tobias Kowatsch, Shafiq Joty, Yin-Leng Theng, and Rifat Atun. 2020. Conversational agents in health care: scoping review and conceptual analysis. Journal of medical Internet research, 22(8):e17158. find
  124. Vasudha Varadarajan, Salvatore Giorgi, Siddharth Mangalik, Nikita Soni, Dave M. Markowitz, and H. Andrew Schwartz. 2025. The consistent lack of variance of psychological factors expressed by LLMs and spambots. In Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect), pages 111–119,… find
  125. Huy Vu, Huy Anh Nguyen, Adithya V Ganesan, Swanie Juhng, Oscar NE Kjell, Joao Sedoc, Margaret L Kern, Ryan L Boyd, Lyle Ungar, H Andrew Schwartz, et al. 2024. Psychadapter: Adapting llm transformers to reflect traits, personality and mental health. arXiv preprint arXiv:2412.16882. — [arXiv:2412.16882](https://arxiv.org/abs/2412.16882) find
  126. Ruijie Xu, Zengzhi Wang, Run-Ze Fan, and Pengfei Liu. 2024. Benchmarking benchmark leakage in large language models. arXiv preprint arXiv:2404.18824. find
  127. Tal Yarkoni. 2010. Personality in 100,000 words: A large-scale analysis of personality and word use among bloggers. Journal of research in personality, 44 3:363–373. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2025-big5-chat-shaping-llm-personalities/