Elamur Research⌕
☰
home / papers / 2026 / Challenging the Myth: A Research Arc on LLMs as Human Simulacra

Challenging the Myth: A Research Arc on LLMs as Human Simulacra

Paper · 2026 · Simon Münker, Achim Rettinger and Damian Trilling

Summary

This position-style paper examines a popular assumption in computational social science: that off-the-shelf, instruction-tuned open-source LLMs, given only light prompting, can stand in for human participants. The authors frame it as a null hypothesis and test it through a sequence of their own studies. The work grew out of the EU-funded TWON project, which aimed to build a digital twin of a social network. Early failures led to two lines of investigation.

  • Psychometric strand: compares LLM answers on questionnaires (MFQ, MFQ-2, HSQ) with human baselines. The analysis moves from simple averages to variance and then to the correlation structure among items.
  • Social-agent strand: checks whether LLM-generated political posts are realistic, using linguistic authenticity measures and detection methods.

Both lines lead to the same conclusion. Prompted models can look plausible at the surface while their internal structure differs from human responses, and their synthetic text remains detectable. Fine-tuning reduces these problems but does not remove them. The authors therefore change the question from whether LLMs can replace people to the specific, validated conditions under which they help research. They propose a validation-first agenda called EVAS.

Key points

  • The research arc covers eight studies from 2023–2026. In an early study, zero-shot annotation of German political tweets produced invented labels and inconsistent outputs for identical inputs.
  • On the MFQ, seven open-source models (7B–176B parameters) prompted with conservative, moderate or liberal personas did not reproduce human ideological patterns. Variance across repetitions ranged from 0.030 to 0.425.
  • On the HSQ:
    • Cronbach's α for the models ranged from 0.008 to 0.617, compared with 0.790–0.841 for humans.
    • Exploratory Graph Analysis did not recover the expected four-factor structure in any model, producing 2–8 communities instead.
  • GPT-3.5-turbo and Mistral-7B posts written in English, German and Dutch showed two biases:
    • English posts were rated as more authentic than Dutch ones.
    • Conservative personas tended to drift toward moderate views.
  • Stated limitations: the social-agent strand covers only English, German and the platform X, and the study uses open-source models only.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vrije Universiteit AmsterdamAmsterdam · Netherlands42 works · 3,372 citations🎓Universität TrierTrier · Germany8 works · 9 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 worksGeneral Social Sciences · 227 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Simon Münker, Achim Rettinger and Damian Trilling. 2026. Challenging the Myth: A Research Arc on LLMs as Human Simulacra. Proceedings of The Big Picture v2: Crafting a Research Narrative.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.bigpicture-main.4

@inproceedings{munker2026challenging,
  title = {{Challenging the Myth: A Research Arc on LLMs as Human Simulacra}},
  author = {Simon Münker, Achim Rettinger and Damian Trilling},
  booktitle = {Proceedings of The Big Picture v2: Crafting a Research Narrative},
  year = {2026},
  doi = {10.18653/v1/2026.bigpicture-main.4},
  url = {https://doi.org/10.18653/v1/2026.bigpicture-main.4}
}

References

92 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. George EP Box. 1976. Science and statistics. Journal of the American Statistical Association, 71(356):791– 799. find
  2. Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298–306.
  3. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  5. Muhammad Farid Adilazuarda, Sagnik Mukherjee, Pradhyumna Lavania, Siddhant Shivdutt Singh, Alham Fikri Aji, Jacki O’Neill, Ashutosh Modi, and Monojit Choudhury. 2024. Towards measuring and modeling “culture” in llms: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Lan…
  6. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, and 1 others. 2023. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint arXiv:2303.12712.
  7. William Agnew, A Stevie Bergman, Jennifer Chien, Mark Díaz, Seliem El-Sayed, Jaylen Pittman, Shakir Mohamed, and Kevin R McKee. 2024. The illusion of artificial inclusion. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1–12. find
  8. Yun-Shiuan Chuang, Agam Goyal, Nikunj Harlalka, Siddharth Suresh, Robert Hawkins, Sijia Yang, Dhavan Shah, Junjie Hu, and Timothy Rogers. 2024. Simulating opinion dynamics with networks of llm-based agents. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3326–3346. find
  9. Meysam Alizadeh, Maël Kubli, Zeynab Samei, Shirin Dehghani, Mohammadmasiha Zahedivafa, Juan D Bermeo, Maria Korobeynikova, and Fabrizio Gilardi. 2025. Open-source llms for text annotation: a practical guide for model setting and fine-tuning. Journal of Computational Social Science, 8(1):17. find
  10. Lee J Cronbach. 1951. Coefficient alpha and the internal structure of tests. psychometrika, 16(3):297–334. find
  11. Adrian de Wynter. 2025. Awes, laws, and flaws from today’s llm research. In Findings of the Association for Computational Linguistics: ACL 2025, pages 12834–12854. find
  12. Barrett R Anderson, Jash Hemant Shah, and Max Kreminski. 2024. Homogenization effects of large language models on human creative ideation. In Proceedings of the 16th conference on creativity & cognition, pages 413–425. find
  13. Dorottya Demszky, Diyi Yang, David S Yeager, Christopher J Bryan, Margarett Clapper, Susannah Chandhok, Johannes C Eichstaedt, Cameron Hecht, Jeremy Jamieson, Meghann Johnson, and 1 others. 2023. Using large language models in psychology. Nature Reviews Psychology, 2(11):688–701. find
  14. Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351.
  15. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  16. Paul Balluff, Justin Chun-ting Ho, Johannes B Gruber, Sean Palicki, Alexis Palmer, Luca Rossi, Irina Shklovski, and Chung-hong Chan. 2026. Newer, larger, better? a critique of the unreflective llm adoption in communication research. Political Communication, pages 1–10. find
  17. Daniel Duncan. 2024. Does chatgpt have sociolinguistic competence? Journal of Computer-Assisted Linguistic Research, 8:51–75. find
  18. Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. 2024. Transcoders find interpretable llm feature circuits. Advances in Neural Information Processing Systems, 37:24375–24410. find
  19. Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social biases in NLP models as barriers for persons with disabilities. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5491–5501.
  20. Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang Lorraine Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, and 1 others. 2024. Faith and fate: Limits of transformers on compositionality. Advances in Neural Information Processing Systems, 36.
  21. Joel Jang, Seonghyeon Ye, and Minjoon Seo. 2023. Can large language models truly understand prompts? a case study with negated prompts. In Transfer Learning for Natural Language Processing Workshop, pages 52–62. PMLR. find
  22. Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2024. Large language models empowered agent-based modeling and simulation: A survey and perspectives. Humanities and Social Sciences Communications, 11(1):1–24. find
  23. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  24. Hudson F Golino and Sacha Epskamp. 2017. Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS one, 12(6):e0174035. find
  25. Karen Sparck Jones. 1994. Natural language processing: a historical review. Current issues in computational linguistics: in honour of Don Walker, pages 3–16. find
  26. Jesse Graham, Jonathan Haidt, and Brian A Nosek. 2009. Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5):1029. find
  27. Jesse Graham, Brian A Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter H Ditto. 2011. Mapping the moral domain. Journal of personality and social psychology, 101(2):366. find
  28. Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, and 1 others. 2021. Wilds: A benchmark of in-the-wild distribution shifts. In International conference on machine learning, pages 5637–… find
  29. Justin Grimmer and Brandon M Stewart. 2013. Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political analysis, 21(3):267–297. find
  30. Maik Larooij and Petter Törnberg. 2025a. Do large language models solve the problems of agent-based modeling? a critical review of generative social simulations. arXiv preprint arXiv:2504.03274. — [arXiv:2504.03274](https://arxiv.org/abs/2504.03274) find
  31. Igor Grossmann, Matthew Feinberg, Dawn C Parker, Nicholas A Christakis, Philip E Tetlock, and William A Cunningham. 2023. Ai and the transformation of social science research. Science, 380(6650):1108–1109. find
  32. Maik Larooij and Petter Törnberg. 2025b. Validation is the central challenge for generative social simulation: a critical review of llms in agent-based modeling. Artificial Intelligence Review, 59(1):15. find
  33. Seungbeen Lee, Seungwon Lim, Seungju Han, Giyeong Oh, Hyungjoo Chae, Jiwan Chung, Minju Kim, Beong-woo Kwak, Yeonsoo Lee, Dongha Lee, and 1 others. 2025. Do llms have distinct and consistent personality? trait: Personality testset designed for llms with psychometrics. In Findings of the Association… find
  34. Peter K Hatemi, Charles Crabtree, and Kevin B Smith. 2019. Ideology justifies morality: Political beliefs predict moral foundations. American Journal of Political Science, 63(4):788–806. find
  35. Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, and 1 others. 2022. Challenges and strategies in cross-cultural nlp. In Proceedings of the 60th Annual Meeting of the Assoc…
  36. Haocheng Lin. 2024. Designing domain-specific large language models: The critical role of fine-tuning in public opinion simulation. arXiv preprint arXiv:2409.19308. — [arXiv:2409.19308](https://arxiv.org/abs/2409.19308) find
  37. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35.
  38. Michael Heseltine. 2025. Comparing large language models for text classification: Model selection across tasks, texts, and languages. find
  39. Ruibo Liu, Chenyan Jia, Jason Wei, Guangxuan Xu, and Soroush Vosoughi. 2022. Quantifying and alleviating political bias in language models. Artificial Intelligence, 304:103654. find
  40. Simon Münker and Fabio Sartori. 2026. Guardrail vulnerabilities in open-source language models: Implications for democratic discourse and marginalized communities. Hawaii International Conference on System Sciences (HICSS). find
  41. Rod A Martin, Patricia Puhlik-Doris, Gwen Larsen, Jeanette Gray, and Kelly Weir. 2003. Individual differences in uses of humor and their relation to psychological well-being: Development of the humor styles questionnaire. Journal of research in personality, 37(1):48–75. find
  42. Simon Münker, Nils Schwager, and Achim Rettinger. 2026. Don’t trust generative agents to mimic communication on social networks unless you benchmarked their empirical realism. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Lo…
  43. John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. 2021. Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In International conference on machine lea… find
  44. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, and 1 others. 2024. Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages. Advances in Neural Information Processing Syste…
  45. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064.
  46. Simon Münker, Nils Schwager, Kai Kugler, Michael Heseltine, and Achim Rettinger. 2026. Next reply prediction x dataset: Linguistic discrepancies in naively generated content. Preprint, arXiv:2602.19177. — [arXiv:2602.19177](https://arxiv.org/abs/2602.19177) find
  47. Hadi Mohammadi, Yasmeen FSS Meijer, Efthymia Papadopoulou, and Ayoub Bagheri. 2025. Do large language models understand morality across cultures? In Proceedings of the 2nd LUHME Workshop, pages 30–39. find
  48. Lynnette Hui Xian Ng and Kathleen M Carley. 2025. Are llm-powered social media bots realistic? In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, pages 14–23. Springer. find
  49. Anders Giovanni Møller and Luca Maria Aiello. 2024. Prompt refinement or fine-tuning? best practices for using llms in computational social science tasks. arXiv preprint arXiv:2408.01346. — [arXiv:2408.01346](https://arxiv.org/abs/2408.01346) find
  50. Megan Norris and Luc Lecavalier. 2010. Evaluating the use of exploratory factor analysis in developmental disability psychological research. Journal of autism and developmental disorders, 40:8–20. find
  51. Simon Münker. 2025a. Cultural bias in large language models: Evaluating ai agents through moral questionnaires. In Proceedings of 0 th Moral and Legal AI Alignment Symposium of the IACAP/AISB Conference, page 61. find
  52. Jum C Nunnally. 1975. Psychometric theory—25 years ago and now. Educational Researcher, 4(10):7–21. find
  53. Étienne Ollion, Rubing Shen, Ana Macanovic, and Arnault Chatelain. 2024. The dangers of using proprietary llms for research. Nature Machine Intelligence, 6(1):4–5. find
  54. Simon Münker. 2025b. Fingerprinting llms through survey item factor correlation: A case study on humor style questionnaire. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 245–258. find
  55. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  56. Simon Münker. 2025c. Political bias in llms: Unaligned moral values in agent-centric simulations. Journal for Language Technology and Computational Linguistics, 38(2):125–138. find
  57. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22. find
  58. Simon Münker, Kai Kugler, and Achim Rettinger. 2025. Zero-shot prompt-based classification: topic labeling in times of foundation models in german tweets. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages 53–63. find
  59. Maja Pavlovic and Massimo Poesio. 2024. The effectiveness of llms as annotators: A comparative overview and empirical analysis of direct representation. In Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives)@ LREC-COLING 2024, pages 100–110.
  60. Simon Münker and Achim Rettinger. 2025. twony: A micro-simulation of the impact of osn mechanics on the emotionality of online discourse. In Joint Proceedings of the ESWC 2025 Workshops and Tutorials. find
  61. Karl Pearson. 1901. Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11):559–572. find
  62. Murray Shanahan. 2024. Simulacra as conscious exotica. Inquiry, pages 1–29. find
  63. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-llm: A trainable agent for role-playing. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153–13187.
  64. Nikolay B Petrov, Gregory Serapio-García, and Jason Rentfrow. 2024. Limited ability of llms to simulate human psychological behaviours: a psychometric analysis. arXiv preprint arXiv:2405.07248. — [arXiv:2405.07248](https://arxiv.org/abs/2405.07248) find
  65. Hua Shen, Tiffany Knearem, Reshmi Ghosh, Kenan Alkiek, Kundan Krishna, Yachuan Liu, Ziqiao Ma, Savvas Petridis, Yi-Hao Peng, Li Qiwei, and 1 others. 2024. Towards bidirectional human-ai alignment: A systematic review for clarifications, framework, and future directions. arXiv preprint arXiv:2406.092… find
  66. Karl Popper. 2014. Conjectures and refutations: The growth of scientific knowledge. routledge. find
  67. Aman Priyanshu and Supriti Vijay. 2024. The silent curriculum: How does llm monoculture shape educational content and its accessibility? arXiv preprint arXiv:2407.10371. — [arXiv:2407.10371](https://arxiv.org/abs/2407.10371) find
  68. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2024. You don’t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. In Proceedings of t… find
  69. Weihong Qi, Hanjia Lyu, and Jiebo Luo. 2025. Representation bias in political sample simulations with large language models. In Companion Proceedings of the ACM on Web Conference 2025, pages 1264– 1267. find
  70. Juan Ramos and 1 others. 2003. Using tf-idf to determine word relevance in document queries. In Proceedings of the first instructional conference on machine learning, volume 242, pages 29–48. New Jersey, USA. find
  71. Lorenz Sparrenberg, Tobias Schneider, Tobias Deußer, Markus Koppenborg, and Rafet Sifa. 2024. Correcting systematic bias in llm-generated dialogues using big five personality traits. In 2024 IEEE International Conference on Big Data (BigData), pages 3061–3069. IEEE. find
  72. Yuqi Ren, Renren Jin, Tongxuan Zhang, and Deyi Xiong. 2025. Do large language models mirror cognitive language processing? In Proceedings of the 31st International Conference on Computational Linguistics, pages 2988–3001.
  73. Sjoerd B Stolwijk, Mark Boukes, Wang Ngai Yeung, Yufang Liao, Simon Münker, Anne C Kroon, and Damian Trilling. 2025. Can we use automated approaches to measure the quality of online political discussion? how to (not) measure interactivity, diversity, rationality, and incivility in online comments to… find
  74. Giulio Rossetti, Massimo Stella, Rémy Cazabet, Katherine Abramski, Erica Cau, Salvatore Citraro, Andrea Failla, Riccardo Improta, Virginia Morini, and Valentina Pansanella. 2024. Y social: an llm-powered social media digital twin. arXiv preprint arXiv:2408.00818. — [arXiv:2408.00818](https://arxiv.org/abs/2408.00818) find
  75. David Rozado. 2023. The political biases of chatgpt. Social Sciences, 12(3):148. find
  76. Amir Taubenfeld, Yaniv Dover, Roi Reichart, and Ariel Goldstein. 2024. Systematic biases in llm simulations of debates. In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 251–267.
  77. Jérôme Rutinowski, Sven Franke, Jan Endendyk, Ina Dormuth, Moritz Roidl, and Markus Pauly. 2024. The self-perception and political biases of Chat-GPT. Human Behavior and Emerging Technologies, 2024(1):7115633. find
  78. Timm Teubner, Christoph M Flath, Christof Weinhardt, Wil van der Aalst, and Oliver Hinz. 2023. Welcome to the era of ChatGPT et al. - the prospects of large language models. Business & Information Systems Engineering, 65(2):95–101. find
  79. Michael Ryan, William Held, and Diyi Yang. 2024. Unintended impacts of llm alignment on global representation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16121–16140.
  80. Surendrabikram Thapa, Shuvam Shiwakoti, Siddhant Bikram Shah, Surabhi Adhikari, Hariram Veeramani, Mehwish Nasim, and Usman Naseem. 2025. Large language models (llm) in computational social science: prospects, current state, and challenges. Social Network Analysis and Mining, 15(1):1–30. find
  81. Arleen Salles, Kathinka Evers, and Michele Farisco. 2020. Anthropomorphism in ai. AJOB neuroscience, 11(2):88–95. find
  82. Jivnesh Sandhan, Fei Cheng, Tushar Sandhan, and Yugo Murawaki. 2025. Cape: Context-aware personality evaluation framework for large language models. Findings of the Association for Computational Linguistics: EMNLP, 2025:10648–10662. find
  83. Lindia Tjuatja, Valerie Chen, Tongshuang Wu, Ameet Talwalkwar, and Graham Neubig. 2024. Do llms exhibit human-like response biases? a case study in survey design. Transactions of the Association for Computational Linguistics, 12:1011–1026.
  84. Petter Törnberg, Diliara Valeeva, Justus Uitermark, and Christopher Bail. 2023. Simulating social media using large language models to evaluate alternative news feed algorithms. arXiv preprint arXiv:2310.05984. — [arXiv:2310.05984](https://arxiv.org/abs/2310.05984) find
  85. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  86. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32.
  87. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP workshop BlackboxNLP: Analyzing and interpreting neural networks for NLP, pages 353– 355…
  88. Angelina Wang, Jamie Morgenstern, and John P Dickerson. 2025. Large language models that replace human participants can harmfully misportray and flatten identity groups. Nature Machine Intelligence, 7(3):400–411. find
  89. Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C. Schmidt. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. In Proceedings of the 30th Conference on Pattern Languages of Programs. The Hillsid…
  90. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. find
  91. Haoran Ye, Jing Jin, Yuhang Xie, Xin Zhang, and Guojie Song. 2025. Large language model psychometrics: A systematic review of evaluation, validation, and enhancement. arXiv preprint arXiv:2505.08245. — [arXiv:2505.08245](https://arxiv.org/abs/2505.08245) find
  92. Ziyun Yu, Yiru Zhou, Chen Zhao, and Hongyi Wen. 2025. An analysis of large language models for simulating user responses in surveys. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Comp… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/munker-2026-challenging-myth-research-arc-llms/