Elamur Research⌕
☰
home / papers / 2026 / Is He Extroverted? Identifying Missing Relevant Personas for Faithful User Simulation

Is He Extroverted? Identifying Missing Relevant Personas for Faithful User Simulation

Paper · 2026 · Weiwen Su, Yuhan Zhou, Zihan Wang et al.

Summary

User simulators built on LLMs usually take the supplied persona as adequate. When a trait that drives a decision is absent, the simulated answer may be plausible but still wrong. This paper treats that gap as a new task: given a dialogue context and a minimal persona (age, gender, relation to the interlocutor), name the persona dimensions that would be needed to predict the user's reply. To test it, the authors created PICQ-drama from TVShowGuess sitcom and drama scripts. It contains persona-influenced choice questions (PICQs) with their preceding context. Candidate questions were found by LLM pre-screening and then checked manually, and annotators described the missing dimensions behind each choice. Outputs are scored on three axes: influence on the choice, inaccessibility (how hard the trait is to obtain), and fidelity to human annotations. The authors also propose a multi-task prompting strategy. Experiments with GPT-4.1, Qwen-3 and Llama-3.1 show the task is solvable, with detection of influential traits improving as models grow. The results also reveal systematic differences between how LLMs and humans pick explanatory personas.

Key points

  • Choice of question type: A manual check of 400 question-mark sentences from TVShowGuess found:
    • 61.1% were fact-seeking and 25.6% were persona-influenced.
    • About 75% of the persona-influenced questions were choice-based, which is why the paper focuses on PICQs.
  • Prompting strategy: The multi-task instructions include two extra stages: first condensing the choice situation into a summary, then abstracting the identified personas into more general forms.
    • In ablations, summarization helped mainly Qwen3-32B (Influence 1.589 with it vs. 1.500 without).
    • Generalization mainly affected GPT-4.1. It raised influence but lowered fidelity (F1 0.300 with it vs. 0.364 without) and reduced the average number of personas (3.38 vs. 4.79).
  • Human vs. LLM patterns: Fidelity to humans follows an inverted U-shaped curve, which the authors link to human "cognitive economy." Humans favour high-impact, easy-to-obtain traits (Personality, Taste, Attribute). GPT-4.1 keeps influence high across categories.
  • Persona categories: Belief, Goal and Relationship are high-impact but hard to acquire. Taste is influential and easy to obtain.
  • Limitations and availability: The data are English-only and come from scripted rather than spontaneous dialogue. The dataset is released at github.com/NioHww/PICQ.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of TokyoTokyo · Japan90 works · 9,156 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksHuman-Computer Interaction · 141 works

Cite

Cite as Weiwen Su, Yuhan Zhou, Zihan Wang, Naoki Yoshinaga and Masashi Toyoda. 2026. Is He Extroverted? Identifying Missing Relevant Personas for Faithful User Simulation. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-srw.9

@inproceedings{su2026is,
  title = {{Is He Extroverted? Identifying Missing Relevant Personas for Faithful User Simulation}},
  author = {Weiwen Su, Yuhan Zhou, Zihan Wang, Naoki Yoshinaga and Masashi Toyoda},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-srw.9},
  url = {https://doi.org/10.18653/v1/2026.eacl-srw.9}
}

References

24 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, and Yanghua Xiao. 2024. From persona to personalization: A survey on role-playing language agents.
  2. Trans. Mach. Learn. Res., 2024. find
  3. Nuo Chen, Yan Wang, Haiyun Jiang, Deng Cai, Yuhan Li, Ziyang Chen, Longyue Wang, and Jia Li. 2023. Large language models meet harry potter: A dataset for aligning dialogue agents with characters. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8506–8520, Singapore. As…
  4. Yun-Shiuan Chuang, Krirk Nirunwiroj, Zach Studdiford, Agam Goyal, Vincent V. Frigo, Sijia Yang, Dhavan V. Shah, Junjie Hu, and Timothy T. Rogers. 2024. Beyond demographics: Aligning role-playing LLM-based agents using human belief networks. In Findings of the Association for Computational Linguistic… find
  5. Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1236–1270, Singapore. Association for Computational Ling…
  6. Chen Gao, Xiaochong Lan, Zhihong Lu, Jinzhu Mao, Jinghua Piao, Huandong Wang, Depeng Jin, and Yong Li. 2023. S3: Social-network simulation system with large language model-empowered agents. Preprint, arXiv:2307.14984. — [arXiv:2307.14984](https://arxiv.org/abs/2307.14984) find
  7. Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. 2024. Better zero-shot reasoning with role-play prompting. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… find
  8. LlamaTeam. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
  9. OpenAI. 2024. Gpt-4 technical report. arXiv:2303.08774.
  10. Preprint, Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, New York, NY… find
  11. Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S. Bernstein. 2024. Generative agent simulations of 1,000 people. Preprint, arXiv:2411.10109. find
  12. QwenTeam. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  13. Nicholas Rescher. 2017. Cognitive economy: The economic dimension of the theory of knowledge. University of Pittsburgh Pre. find
  14. Yisi Sang, Xiangyang Mou, Mo Yu, Shunyu Yao, Jing Li, and Jeffrey Stanton. 2022. TVShowGuess: Character comprehension in stories as speaker guessing. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pag… find
  15. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-LLM: A trainable agent for role-playing. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153–13187, Singapore. Association for Computational Linguistics.
  16. Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. 2024. CharacterEval: A Chinese benchmark for role-playing conversational agent evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1183…
  17. Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. 2024. Large language models are superpositions of all characters: Attaining arbitrary role-play via self-alignment. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7828–7840,…
  18. Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, and Junran Peng. 2024. RoleLLM: Benchmarking, eliciting, and enhancing role-playing abilities of large la…
  19. Man Tik Ng, Hui Tung Tse, Jen tse Huang, Jingjing Li, Wenxuan Wang, and Michael R. Lyu. 2024. How well can llms echo us? evaluating ai chatbots’ role-play ability with echo. Preprint, arXiv:2404.13957. — [arXiv:2404.13957](https://arxiv.org/abs/2404.13957) find
  20. Xiaoyang Wang, Hongming Zhang, Tao Ge, Wenhao Yu, Dian Yu, and Dong Yu. 2025. Opencharacter: Training customizable role-playing llms with large-scale synthetic personas. Preprint, arXiv:2501.15427. occupation, or cultural background. These factors may constrain or inform choices due to physical capa… — [arXiv:2501.15427](https://arxiv.org/abs/2501.15427) find
  21. Goals Current intentions, needs, or objectives a person is trying to achieve. Goals directly impact choices by framing what is desirable or prioritized in a given situation. find
  22. Sanae Yamashita, Koji Inoue, Ao Guo, Shota Mochizuki, Tatsuya Kawahara, and Ryuichiro Higashinaka. 2023. RealPersonaChat: A realistic persona chat corpus with interlocutors’ own personalities. In Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation, pages 852–861,… find
  23. Xinfeng Yuan, Siyu Yuan, Yuhan Cui, Tianhe Lin, Xintao Wang, Rui Xu, Jiangjie Chen, and Deqing Yang. 2024. Evaluating character understanding of large language models via character profiling from fictional works. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processi… find
  24. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbou…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/su-2026-is-he-extroverted-identifying-missing/