Elamur Research⌕
☰
home / papers / 2026 / Generative Personality Simulation via Theory-Informed Structured Interview

Generative Personality Simulation via Theory-Informed Structured Interview

Paper · 2026 · Pengda Wang, Huiqi Zou, Han Jiang et al.

Summary

LLMs are increasingly used as stand-ins for human participants, but their simulated answers to personality questionnaires tend to be too uniform. They miss the item-level variation and heterogeneity seen in real people, which limits their usefulness for social science. The authors address this with the Personality Structured Interview (PSI), a set of interview questions built with psychometric scale-development methods and grounded in the Five Factor Model. Instead of turning scale items into yes/no questions, PSI asks behavioral, narrative prompts. The transcripts of human answers are then given to an LLM, which responds to personality inventories (BFI-2) and behavior items (counterproductive work behavior) as that person would.

Evaluation follows measurement theory, checking reliability, structural (factor) validity and external validity at the item, facet and domain levels. Across three experiments, PSI-conditioned simulations correlated moderately and significantly with participants' own self-reports. They also reproduced population-level heterogeneity and factor structure better than competing approaches. Simulated behavior followed expected trends but looked more idealized than real human behavior. The work offers a psychometrically sound route to synthetic data for hard-to-collect populations.

Key points

  • Released resource: a dataset of 357 structured interview transcripts, with code and data at github.com/isle-dev/PSI. Personally identifiable information was removed, and data collection was IRB-approved.
  • Three experiments:
    • reproducing individual-level BFI-2 responses;
    • matching human personality distributions, including descriptive statistics and factor analysis;
    • predicting personality-related behaviors such as counterproductive work behavior.
  • Evaluation framework: a measurement-theory-based procedure (Cronbach; Cronbach & Meehl) that respects the item-to-latent-trait hierarchy and tests reliability, structural validity and external validity.
  • Question ablation: removing single questions lowered R² only slightly, typically within ±0.01. The most important questions differed by target domain, which suggests the questions carry complementary information.
  • Limitations: the tools are human self-report measures, and LLMs do not have a personality themselves. Simulated behaviors remain more idealized than human ones, and generalization to other constructs still needs testing.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Pengda Wang, Huiqi Zou, Han Jiang, Hanjie Chen, Tianjun Sun, Xiaoyuan Yi, Ziang Xiao and Frederick L. Oswald. 2026. Generative Personality Simulation via Theory-Informed Structured Interview. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.82

@inproceedings{wang2026generative,
  title = {{Generative Personality Simulation via Theory-Informed Structured Interview}},
  author = {Pengda Wang, Huiqi Zou, Han Jiang, Hanjie Chen, Tianjun Sun, Xiaoyuan Yi, Ziang Xiao and Frederick L. Oswald},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.82},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.82}
}

References

64 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024. Llama 3 model card. find
  2. Jüri Allik, Anu Realo, René Mõttus, and Peter Kuppens. 2010. Generalizability of self–other agreement from one personality trait to another. Personality and Individual Differences, 48(2):128–132. find
  3. Gordon W Allport and Henry S Odbert. 1936. Trait-names: A psycho-lexical study. The Psychological Monographs, 47(1):i. find
  4. Christopher M Berry, Deniz S Ones, and Paul R Sackett. 2007. Interpersonal deviance, organizational deviance, and their common correlates: A review and meta-analysis. Journal of Applied Psychology, 92(2):410. find
  5. Michael A Campion, David K Palmer, and James E Campion. 1997. A review of structure in the selection interview. Personnel Psychology, 50(3):655–702. find
  6. James J Connolly, Erin J Kavanagh, and Chockalingam Viswesvaran. 2007. The convergent validity between self and observer ratings of personality: A meta-analytic review. International Journal of Selection and Assessment, 15(1):110–117. find
  7. Paul T Costa and Robert R McCrae. 2008. The revised neo personality inventory (neo-pi-r). The SAGE handbook of personality theory and assessment, 2(2):179– 198. find
  8. Marcus Credé, Michael C Tynan, and Peter D Harms. 2017. Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology, 113(3):492. find
  9. Lee J Cronbach. 1951. Coefficient alpha and the internal structure of tests. psychometrika, 16(3):297–334. find
  10. Lee J Cronbach and Paul E Meehl. 1955. Construct validity in psychological tests. Psychological Bulletin, 52(4):281. find
  11. Andrew Cutler and David M Condon. 2023. Deep lexical hypothesis: Identifying personality structure in natural language. Journal of Personality and Social Psychology, 125(1):173. find
  12. Rodica Ioana Damian, Martin Spengler, Andreea Sutu, and Brent W. Roberts. 2019. Sixteen going on sixty-six: A longitudinal study of personality stability and change across 50 years. Journal of Personality and Social Psychology, 117(3):674–695. find
  13. Francis Galton. 1884. Measurement of character. Fortnightly, 36(212):179–185. find
  14. Lewis R Goldberg. 1990. An alternative" description of personality": The big-five factor structure. Journal of Personality and Social Psychology, 59(6):1216– 1229. find
  15. Lewis R Goldberg. 1992. The development of markers for the big-five factor structure. Psychological Assessment, 4(1):26. find
  16. Robert Hogan, Joyce Hogan, and Brent W Roberts. 1996. Personality measurement and employment decisions: Questions and answers. American Psychologist, 51(5):469. find
  17. Leaetta M Hough, Frederick L Oswald, and Jisoo Ock. 2015. Beyond the big five: New directions for personality research and practice in organizations. Annu. Rev. Organ. Psychol. Organ. Behav., 2(1):183–209. find
  18. Li-tze Hu and Peter M Bentler. 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1):1–55. find
  19. Jen-tse Huang, Wenxuan Wang, Man Ho Lam, Eric John Li, Wenxiang Jiao, and Michael R Lyu. 2023. Revisiting the reliability of psychological scales on large language models. arXiv preprint arXiv:2305.19926. — [arXiv:2305.19926](https://arxiv.org/abs/2305.19926) find
  20. Mikhila N Humbad, M Brent Donnellan, William G Ia-cono, Matthew McGue, and S Alexandra Burt. 2013. Quantifying the association between personality similarity and marital adjustment using profile correlations: A cautionary tale. Journal of Research in Personality, 47(1):97–106. find
  21. Paul Irwing, David J Hughes, Alexander Tokarev, and Tom Booth. 2024. Towards a taxonomy of personality facets. European Journal of Personality, 38(3):494–515. find
  22. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  23. OP John. 1999. The big-five trait taxonomy: History, measurement, and theoretical perspectives. Handbook of Personality: Theory and Research/Guilford. find
  24. Karl G Jöreskog. 1969. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2):183–202. find
  25. Dana L Joseph and Daniel A Newman. 2010. Emotional intelligence: An integrative meta-analysis and cascading model. Journal of Applied Psychology, 95(1):54. find
  26. Timothy A Judge, Joyce E Bono, Remus Ilies, and Megan W Gerhardt. 2002. Personality and leadership: A qualitative and quantitative review. Journal of Applied Psychology, 87(4):765. find
  27. Alexander Kachur, Evgeny Osin, Denis Davydov, Konstantin Shutilov, and Alexey Novokshonov. 2020. Assessing the big five personality traits using real-life static facial images. Scientific Reports, 10(1):8487. find
  28. Kibeom Lee and Michael C Ashton. 2004. Psychometric properties of the hexaco personality inventory. Multivariate Behavioral Research, 39(2):329–358. find
  29. Kibeom Lee and Michael C Ashton. 2006. Further assessment of the hexaco personality inventory: two new facet scales and an observer report form. Psychological Assessment, 18(2):182. find
  30. Seungbeen Lee, Seungwon Lim, Seungju Han, Giyeong Oh, Hyungjoo Chae, Jiwan Chung, Minju Kim, Beong-woo Kwak, Yeonsoo Lee, Dongha Lee, Jinyoung Yeo, and Youngjae Yu. 2024. Do llms have distinct and consistent personality? trait: Personality testset designed for llms with psychometrics. arXiv preprint… find
  31. Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of Psychology. find
  32. Urbano Lorenzo-Seva and Jos MF Ten Berge. 2006. Tucker’s congruence coefficient as a meaningful index of factor similarity. Methodology, 2(2):57–64. find
  33. Donald R Lynam and Thomas A Widiger. 2001. Using the five-factor model to represent the dsm-iv personality disorders: An expert consensus approach. Journal of Abnormal Psychology, 110(3):401. find
  34. Donald W MacKinnon. 1944. The structure of personality. Personality and the Behavior Disorders. find
  35. Dan P McAdams. 1995. What do we know when we know a person? Journal of Personality, 63(3):365– 396. find
  36. Dan P McAdams. 1996. Personality, modernity, and the storied self: A contemporary framework for studying persons. Psychological Inquiry, 7(4):295–321. find
  37. Dan P McAdams. 2001. The psychology of life stories. Review of General Psychology, 5(2):100–122. find
  38. Robert R McCrae. 2008. A note on some measures of profile agreement. Journal of Personality Assessment, 90(2):105–109. find
  39. Robert R McCrae and Paul T Costa Jr. 1997. Personality trait structure as a human universal. American psychologist, 52(5):509. find
  40. Lisa Messeri and MJ Crockett. 2024. Artificial intelligence and illusions of understanding in scientific research. Nature, 627(8002):49–58. find
  41. Warren T Norman. 1963. Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. The Journal of Abnormal and Social Psychology, 66(6):574. find
  42. Deniz S Ones and Brenton M. Wiernik. 2018. On “new” personality types. find
  43. Brent W Roberts, Nathan R Kuncel, Rebecca Shiner, Avshalom Caspi, and Lewis R Goldberg. 2007. The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological Science, 2(4):313–… find
  44. Brent W Roberts, Kate E Walton, and Wolfgang Viecht-bauer. 2006. Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Psychological Bulletin, 132(1):1. find
  45. Richard W Robins, Avshalom Caspi, and Terrie E Moffitt. 2002. It’s not just who you’re with, it’s who you are: Personality and relationship experiences across multiple relationships. Journal of Personality, 70(6):925–964. find
  46. Gerard Saucier and Lewis R Goldberg. 2001. Lexical studies of indigenous personality factors: Premises, products, and prospects. Journal of Personality, 69(6):847–879. find
  47. Greg Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2023. Personality traits in large language models. arXiv preprint arXiv:2307.00184.
  48. Christopher J Soto and Oliver P John. 2017a. The next big five inventory (bfi-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1):117. find
  49. Christopher J Soto and Oliver P John. 2017b. Short and extra-short forms of the big five inventory–2: The bfi-2-s and bfi-2-xs. Journal of Research in Personality, 68:69–81. find
  50. Paul E Spector and Suzy Fox. 2010. Counterproductive work behavior and organisational citizenship behavior: Are they opposite forms of active behavior? Applied Psychology, 59(1):21–39. find
  51. OpenAI. 2024b. Hello gpt-4o. find
  52. Timothy J Trull, Thomas A Widiger, J David Useda, Jay Holcomb, Bao-Tran Doan, Seth R Axelrod, Barry L Stern, and Beth S Gershuny. 1998. A structured interview for the assessment of the five-factor model of personality. Psychological Assessment, 10(3):229. find
  53. Dennis W Organ and Katherine Ryan. 1995. A meta-analytic review of attitudinal and dispositional predictors of organizational citizenship behavior. Personnel Psychology, 48(4):775–802. find
  54. L. R. Tucker. 1951. A method for synthesis of factor analysis studies. Personnel Research Section Report 984, Personnel Research Section, Department of the Army. find
  55. Jeongeon Park, Bryan Min, Xiaojuan Ma, and Juho Kim. 2023. Choicemates: Supporting unfamiliar online decision-making with multi-agent conversational interactions. arXiv preprint arXiv:2310.01331. — [arXiv:2310.01331](https://arxiv.org/abs/2310.01331) find
  56. Pengda Wang, Huiqi Zou, Zihan Yan, Feng Guo, Tianjun Sun, Ziang Xiao, and Bo Zhang. 2024. Not yet: Large language models cannot replace human respondents for psychometric research. OSF. find
  57. Joon Sung Park, Carolyn Q Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S Bernstein. 2024. Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109. find
  58. Xintao Wang, Quan Tu, Yaying Fei, Ziang Leng, and Cheng Li. 2023. Does role-playing chatbots capture the character personalities? assessing personality traits for role-playing chatbots. arXiv e-prints, pages arXiv–2310. find
  59. OpenAI. 2024a. Gpt-4o mini: advancing cost-efficient intelligence. find
  60. Michael P Wilmot. 2015. A contemporary taxometric analysis of the latent structure of self-monitoring. Psychological Assessment, 27(2):353. find
  61. Michael P Wilmot, Nick Haslam, Jingyuan Tian, and Deniz S Ones. 2019. Direct and conceptual replications of the taxometric analysis of type a behavior. Journal of Personality and Social Psychology, 116(3):e12. find
  62. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, and 1 others. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864.
  63. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243.
  64. Content of Appendix A Appendix: BFI-2 Scale and Behavioral Measures Scales A.1 BFI-2 Scale . . . . . . . . . . . . A.2 Behavioral Measures Scales . . . I.2 15 15 15 B Appendix: Development and Validation of Personality-Structured Interview Questions 16 B.1 Questions and Development Process 16 B.2 Va… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-generative-personality-simulation-theory-informed/