Elamur Research⌕
☰
home / papers / 2024 / LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals

LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals

Paper · 2024 · Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst et al.

Summary

Machine-learning models that predict human behavior usually need large amounts of structured data for each specific outcome, so they are hard to reuse for new questions. The authors test whether LLM-based "generative agents" conditioned on people's self-reports can predict an individual's responses across many unrelated tasks without any task-specific training. They recruited a stratified sample of 1,052 U.S. adults. Each person spoke for about two hours with an AI voice interviewer that followed the American Voices Project protocol. Participants then completed the General Social Survey core, the BFI-44, incentivized economic games and replicated experiments, and repeated these measures two weeks later. For every person, the authors built agents from the interview transcript, from the survey answers, or from both. They compared these with demographics-only and persona baselines and scored accuracy relative to each participant's own test-retest consistency. Agents grounded in self-reports clearly outperformed demographic prompting, and the combined variant scored best, though its gain over either source alone was small. The results suggest that rich self-reports, which AI interviewers can collect at scale, enable reusable individual-level simulation with smaller accuracy gaps across racial and ideological groups.

Key points

  • Data: 1,052 participants, recruited so that the sample met target proportions by political party, educational attainment, region, race, gender and age. Interview transcripts averaged 6,491 words (std 2,541).
  • GSS results (normalized accuracy): interview agents scored 0.83, survey agents 0.82 and survey-plus-interview agents 0.86. Demographics-only agents scored 0.74 and persona-description agents 0.71. In raw terms, combined agents reached 68.30% accuracy, while participants matched their own earlier answers 79.53% of the time.
  • Big Five: interview agents reached a normalized correlation of 0.80, compared with 0.61 for demographics-only agents.
  • Economic games: the five games were the dictator game, both roles of the Trust Game, the public goods game and the prisoner's dilemma. Interview agents achieved a normalized correlation of 0.66, compared with 0.47 for demographic agents. Survey-based agents did relatively poorly here (0.38).
  • Experiments and evaluation method: five text-based experiments were taken from a large-scale replication effort. Test-retest normalization is proposed as a benchmark that allows comparison across instruments with different reliability.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States492 works · 440,463 citations🎓Stanford UniversityStanford · United States506 works · 330,053 citations🎓University of WashingtonSeattle · United States402 works · 212,763 citations🏢Google DeepMindLondon · United Kingdom72 works · 84,326 citations🎓Northwestern UniversityEvanston · United States46 works · 2,850 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 worksGeneral Social Sciences · 254 worksBusiness, Management and Accounting · 40 worksOrganizational Behavior and Human Resource Management · 15 works

Cite

Cite as Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer and Michael S. Bernstein. 2024. LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals. arXiv:2411.10109.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2411.10109 · arXiv 2411.10109

@misc{park2024llm,
  title = {{LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals}},
  author = {Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer and Michael S. Bernstein},
  year = {2024},
  doi = {10.48550/arxiv.2411.10109},
  eprint = {2411.10109},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2411.10109}
}

References

7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 1.​ A. Ashokkumar, L. Hewitt, I. Ghezae, R. Willer, "Predicting Results of Social Science Experiments Using Large Language Models" (2024). 2.​ L. P. Argyle, E. C. Busby, N. Fulda, J. R. Gubler, C. Rytting, D. Wingate, Out of one, many: Using language models to simulate human samples. Political Analy… find
  2. M., Zhang, X., Soto, C., & Evans, J. (2026, January 9). Designing AI-agents with personalities: A Psychometric Approach - Muhua Huang, Xijuan Zhang, Christopher Soto, James Evans, 2026. Personality Science. https://journals.sagepub.com/doi/10.1177/27000710251406471 11.M. J. Salganik et al., Measuri… ↗
  3. Barocas, M. Hardt, A. Narayanan, Fairness and Machine Learning (2019); https://fairmlbook.org 46.Gao et al. 2025, PNAS — “Take caution in using LLMs as human surrogates.” 47.Dominguez-Olmedo, Hardt, and Mendler-Dünner 2024, NeurIPS — “Questioning the Survey Responses of Large Language Models.” 48.… — [doi:10.1017/pan.2024.5](https://doi.org/10.1017/pan.2024.5) ↗
  4. PsyArXiv. 2023. 51.M. N. Stagnaro, J. Druckman, A. J. Berinsky, A. A. Arechar, R. Willer, D. Rand, Representativeness versus Response Quality: Assessing Nine Opt-In Online Survey Samples. OSF Preprints [Preprint] (2024); https://osf.io/preprints/psyarxiv/h9j2dc 52.E. Bruch, J. Atwell, Agent-Based… ↗
  5. C. Schelling, Micromotives and Macrobehavior (W. W. Norton & Company, 1978). 59.E. Bonabeau, Agent-based modeling: Methods and techniques for simulating human systems. Proc. Natl. Acad. Sci. U.S.A. 99 (suppl. 3), 7280-7287 (2002); https://doi.org/10.1073/pnas.082080899 60.M. W. Macy, R. Willer, Fr… — [doi:10.1073/pnas.082080899](https://doi.org/10.1073/pnas.082080899) ↗
  6. Reeves, C. Nass, The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places (Cambridge University Press, 1996). 69.OpenAI, "Text to speech guide" (2024); https://platform.openai.com/docs/guides/text-to-speech (accessed 20 September 2024). 70.OpenAI, "Whis… — [doi:10.31234/osf.io/37w8p](https://doi.org/10.31234/osf.io/37w8p) ↗
  7. Turitto, G. VandenBos, S. Vazire, E. J. Wagenmakers, R. Wilson, T. Yarkoni, Promoting an open research culture. Science 348, 1422-1425 (2015). 78.​J. Chandler, M. Cumpston, T. Li, M. J. Page, V. J. H. W. Welch, Cochrane Handbook for Systematic Reviews of Interventions (Wiley, ed. 2, 2019). 79.​Silve…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/park-2024-llm-agents-grounded-self-reports/