Elamur Research⌕
☰
home / papers / 2026 / \textitOne Persona, \textitMany Cues, \textitDifferent Results: How Sociodemographic Cues Impact LLM Personalization

\textitOne Persona, \textitMany Cues, \textitDifferent Results: How Sociodemographic Cues Impact LLM Personalization

Paper · 2026 · Franziska Weeber, Vera Neplenbroek, Jan Batzner et al.

Summary

Studies of bias in LLM personalization usually tell the model who the user is through a single signal, or "persona cue," such as a name or a direct statement of the user's identity. This ignores two issues. LLMs are sensitive to how prompts are worded. Some cues, such as explicit identity statements, rarely appear in real conversations, which weakens external validity.

The authors built a flexible benchmark to test how far bias findings depend on the chosen cue. It covers:

  • six cues, from three families: names, explicit mentions and conversation history;
  • ten single-attribute personas based on gender, race/ethnicity and age;
  • four English writing and advice tasks;
  • seven open and proprietary LLMs.

Average responses per attribute correlate strongly across cues and models. Even so, different cues produce noticeably different gaps between personas, enough to change conclusions about bias. No cue consistently mimics real human conversation histories. Highly explicit cues produce stronger personalization bias. The authors recommend that researchers and engineers base bias claims on several cues, prompt variants and tasks.

Key points

  • Cue design: there are three cue families, each in two variants (e.g., name-system vs. name-user). A no-persona baseline is included. Human-written conversation histories are treated as the most externally valid cue.
  • Personas:
    • Gender: three identities are represented, namely women, men and non-binary people.
    • Race/ethnicity: four groups are included, namely White, Hispanic, Asian and Black users.
    • Age: binned for analysis into 18-34, 35-54 and 55+.
    • Each persona carries only one attribute, so intersectional effects are not studied.
  • Name selection: names come from North Carolina voter registration data, following Rosenman et al. (2023).
    • A name was used only if at least 90% of its holders share the target attribute.
    • Each persona gets five names, inserted into five phrase templates.
    • Gender-ambiguous names are labelled "unisex" rather than non-binary.
  • Tasks and models: the tasks draw on sources such as IssueBench and data from Kearney et al. (2025), and include one open-ended task scored by an LLM-as-a-judge. Gemma-3-27B is among the models tested.
  • Availability: the code and benchmark are published at github.com/frawee/persona_cues.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AmsterdamAmsterdam · Netherlands144 works · 173,906 citations🎓University of StuttgartStuttgart · Germany91 works · 46,376 citations🎓Technical University of MunichMunich · Germany55 works · 48,474 citations🎓Munich Center for Machine LearningMunich · Germany57 works · 1,094 citations

Also on the byline: Weizenbaum Institute.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Franziska Weeber, Vera Neplenbroek, Jan Batzner and Sebastian Padó. 2026. \textitOne Persona, \textitMany Cues, \textitDifferent Results: How Sociodemographic Cues Impact LLM Personalization. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.2079

@inproceedings{weeber2026textitone,
  title = {{textitOne Persona, textitMany Cues, textitDifferent Results: How Sociodemographic Cues Impact LLM Personalization}},
  author = {Franziska Weeber, Vera Neplenbroek, Jan Batzner and Sebastian Padó},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2079},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2079}
}

References

66 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Meta AI. 2025. Llama 3.3 model card. https://www.llama.com/docs/ model-cards-and-prompt-formats/llama3_3/. Model card for the Llama 3.3 language model. ↗
  2. Areej Alhassan, Jinkai Zhang, and Viktor Schlegel. 2022. ‘am I the bad one’? predicting the moral judgement of the crowd using pre–trained language models. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 267–276, Marseille, France. European Language Resources Ass… find
  3. Haozhe An, Christabel Acquaye, Colin Wang, Zongxia Li, and Rachel Rudinger. 2024. Do large language models discriminate in hiring decisions on the basis of race, ethnicity, and gender? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)… find
  4. Jisun An, Haewoon Kwak, and Bernard J Jansen. 2017. Personas for content creators via decomposed aggregate audience statistics. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, pages 632–635. find
  5. Elisa Bassignana, Amanda Cercas Curry, and Dirk Hovy. 2025. The AI gap: How socioeconomic status affects language technology interactions. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18647–18664, Vienna, Austria. Associati…
  6. Jan Batzner, Volker Stocker, Bingjun Tang, Anusha Natarajan, Qinhao Chen, Stefan Schmid, and Gjergji Kasneci. 2025. Whose personae? synthetic persona experiments in llm research and pathways to transparency. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. find
  7. Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou, Franziska Sofia Hafner, Harry Mayne, Jan Batzner, Negar Foroutan, Chris Schmitz, Karolina Korgul, Hunar Batra, Oishi Deb, Emma Beharry, Cornelius Emde, Thomas Foster, Anna Gausen, María Grandury, Simeng Han, Valentin Hofmann, Lujain Ibrahim, and… find
  8. Donald T. Campbell and Donald W. Fiske. 1959. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2):81–105. find
  9. Donald T. Campbell and Julian C. Stanley. 2015. Experimental and Quasi-Experimental Designs for Research. Ravenio Books. find
  10. Tanise Ceron, Neele Falk, Ana Barić, Dmitry Nikolaev, and Sebastian Padó. 2024. Beyond Prompt Brittleness: Evaluating the Reliability and Consistency of Political Worldviews in LLMs. Transactions of the Association for Computational Linguistics, 12:1378– 1400. Place: Cambridge, MA Publisher: MIT Pr…
  11. Aaron Chatterji, Thomas Cunningham, David J Deming, Zoe Hitzig, Christopher Ong, Carl Yan Shan, and Kevin Wadman. 2025. How people use Chat-GPT. Technical report, National Bureau of Economic Research. find
  12. Yida Chen, Aoyu Wu, Trevor DePodesta, Catherine Yeh, Kenneth Li, Nicholas Castillo Marin, Oam Patel, Jan Riecke, Shivam Raval, Olivia Seow, Martin Wattenberg, and Fernanda Viégas. 2024. Designing a dashboard for transparency and control of conversational AI. Preprint, arXiv:2406.07882. — [arXiv:2406.07882](https://arxiv.org/abs/2406.07882) find
  13. Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023. Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1504–1532, Toronto, Canada. Associatio…
  14. Kimberle Crenshaw. 1989. Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics. University of Chicago Legal Forum, 1989(1):139–167. find
  15. Esin Durmus, Karina Nguyen, Thomas Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2024. Towards measuring the…
  16. Bufan Gao and Elisa Kreiss. 2025. Measuring bias or measuring the task: Understanding the brittle nature of LLM gender biases. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6745–6761, Suzhou, China. Association for Computational Linguistics. find
  17. Vagrant Gautam, Arjun Subramonian, Anne Lauscher, and Os Keyes. 2024. Stop! In the Name of Flaws: Disentangling Personal Names and Sociodemographic Attributes in NLP. In Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 323–337, Bangkok, Thailand. Associat… find
  18. Adam Karvonen and Samuel Marks. 2025. Robustly improving LLM fairness in realistic settings via interpretability. Preprint, arXiv:2506.10922. — [arXiv:2506.10922](https://arxiv.org/abs/2506.10922) find
  19. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… find
  20. Hannah Rose Kirk, Alexander Whitefield, Paul Röttger, Andrew Bean, Katerina Margatina, Juan Ciro, Rafael Mosquera, Max Bartolo, Adina Williams, He He, Bertie Vidgen, and Scott A. Hale. 2024. The prism alignment dataset: What participatory, representative and individualised human feedback reveals abo… find
  21. Salvatore Giorgi, Tingting Liu, Ankit Aich, Kelsey Jane Isman, Garrick Sherman, Zachary Fried, João Sedoc, Lyle Ungar, and Brenda Curtis. 2024. Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personas. In Findings of the Association for Computational Linguistics: EMN… find
  22. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — The Llama 3 Herd of Models
  23. Bowen Jiang, Yuan Yuan, Maohao Shen, Zhuoqun Hao, Zhangchen Xu, Zichen Chen, Ziyi Liu, Anvesh Rao Vijjini, Jiashu He, Hanchao Yu, Radha Poovendran, Gregory Wornell, Lyle Ungar, Dan Roth, Sihao Chen, and Camillo Jose Taylor. 2025. Personamem-v2: Towards personalized intelligence via learning implicit… — [arXiv:2512.06688](https://arxiv.org/abs/2512.06688) find
  24. Soon-Gyo Jung, Jisun An, Haewoon Kwak, Moeed Ahmad, Lene Nielsen, and Bernard J Jansen. 2017. Persona generation from aggregated social media data. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems, pages 1748–1755. find
  25. Mahammed Kamruzzaman and Gene Louis Kim. 2025. The impact of name age perception on job recommendations in LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 15033–15058, Vienna, Austria. Association for Computational Linguistics. find
  26. Anjali Kantharuban, Jeremiah Milbauer, Maarten Sap, Emma Strubell, and Graham Neubig. 2025. Stereotype or personalization? user identity biases chatbot recommendations. In Findings of the Association for Computational Linguistics: ACL 2025, pages 24418–24436, Vienna, Austria. Association for Computa… find
  27. Matthew Kearney, Reuben Binns, and Yarin Gal. 2025. Language models change facts based on the way you talk. Preprint, arXiv:2507.14238. — [arXiv:2507.14238](https://arxiv.org/abs/2507.14238) find
  28. Clyde Young Kramer. 1956. Extension of multiple range tests to group means with unequal numbers of replications. Biometrics, 12(3):307–310. find
  29. Do Xuan Long, Kenji Kawaguchi, Min-Yen Kan, and Nancy Chen. 2025. Aligning Large Language Models with Human Opinions through Persona Selection and Value–Belief–Norm Reasoning. In Proceedings of the 31st International Conference on Computational Linguistics, pages 2526–2547, Abu Dhabi, UAE. Associati…
  30. Manikanta Loya, Divya Sinha, and Richard Futrell. 2023. Exploring the sensitivity of LLMs’ decision-making capabilities: Insights from prompt variations and hyperparameters. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3711–3716, Singapore. Association for Computat… find
  31. Marlene Lutz, Indira Sen, Georg Ahnert, Elisa Rogers, and Markus Strohmaier. 2025. The prompt makes the person(a): A systematic evaluation of sociodemographic persona prompting for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23212–23237, Suz… find
  32. Tomasz Miaskiewicz and Kenneth A Kozar. 2011. Personas and user-centered design: How can personas benefit product design processes? Design studies, 32(5):417–430. find
  33. Kirsten Morehouse, Siddharth Swaroop, and Weiwei Pan. 2025. Position: Rethinking LLM bias probing using lessons from the social sciences. In Forty-second International Conference on Machine Learning Position Paper Track. find
  34. Vera Neplenbroek, Arianna Bisazza, and Raquel Fernández. 2025. Reading between the prompts: How stereotypes shape LLM’s implicit personalization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20378–20411, Suzhou, China. Association for Computational…
  35. Anna Neumann, Elisabeth Kirsten, Muhammad Bilal Zafar, and Jatinder Singh. 2025. Position is power: System prompts as a mechanism of bias in large language models (LLMs). In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’25, page 573–598, New York, NY, U… find
  36. Huy Nghiem, John Prindle, Jieyu Zhao, and Hal Daumé Iii. 2024. “You Gotta be a Doctor, Lin”: An investigation of name-based bias of large language models in employment recommendations. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7268– 7287, Miami,… find
  37. Chimaobi Okite, Naihao Deng, Kiran Bodipati, Huaid-ian Hou, Joyce Chai, and Rada Mihalcea. 2025. Benchmarking and improving LLM robustness for personalized generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 16040–16072, Suzhou, China. Association for Computat… find
  38. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  39. Siddhesh Milind Pawar, Arnav Arora, Lucie-Aimée Kaffee, and Isabelle Augenstein. 2025. Presumed cultural identity: How names shape LLM responses. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 22147–22172, Suzhou, China. Association for Computational Linguistics. find
  40. Giulio Pelosio, Devesh Batra, Noémie Bovey, Robert Hankache, Cristovao Iglesias, Greig Cowan, and Raad Khraishi. 2025. Obscured but not erased: Evaluating nationality bias in LLMs via name-based bias benchmarks. Preprint, arXiv:2507.16989. — [arXiv:2507.16989](https://arxiv.org/abs/2507.16989) find
  41. Flor Miriam Plaza-del Arco, Paul Röttger, Nino Scherrer, Emanuele Borgonovo, Elmar Plischke, and Dirk Hovy. 2025. No for some, yes for others: Persona prompts and other sources of false refusal in language models. In Proceedings of the 9th Widening NLP Workshop, pages 268–282, Suzhou, China. Associa… find
  42. Elinor Poole-Dayan, Deb Roy, and Jad Kabbara. 2025. LLM targeted underperformance disproportionately impacts vulnerable users. In Proceedings of NeurIPS. find
  43. Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, and 24 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15… find
  44. Inioluwa Deborah Raji, Emily M. Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the Whole Wide World Benchmark. In Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks. find
  45. Elisa Forcada Rodríguez, Olatz Perez-de Vinaspre, Jon Ander Campos, Dietrich Klakow, and Vagrant Gautam. 2025. Colombian waitresses y jueces cana-dienses: Gender and country biases in occupation recommendations from LLMs. In Proceedings of the 6th Workshop on Gender Bias in Natural Language Processi… find
  46. Evan T. R. Rosenman, Santiago Olivella, and Kosuke Imai. 2023. Race and ethnicity data for first, middle, and surnames. Scientific Data, 10(1):299. find
  47. Paul Röttger, Musashi Hinck, Valentin Hofmann, Kobi Hackenburg, Valentina Pyatkin, Faeze Brahman, and Dirk Hovy. 2026. Issuebench: Millions of realistic prompts for measuring issue bias in llm writing assistance. Transactions of the Association for Computational Linguistics, 14:318–340.
  48. Paul Röttger, Valentin Hofmann, Valentina Pyatkin, Musashi Hinck, Hannah Kirk, Hinrich Schuetze, and Dirk Hovy. 2024. Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models. In Proceedings of the 62nd Annual Meeting of the Associatio…
  49. Joni Salminen, Kathleen Guan, Soon-gyo Jung, Shammur A Chowdhury, and Bernard J Jansen. 2020a. A literature review of quantitative persona creation. In Proceedings of the 2020 CHI conference on human factors in computing systems, pages 1–14. find
  50. Joni Salminen, Joao M Santos, Haewoon Kwak, Jisun An, Soon-gyo Jung, and Bernard J Jansen. 2020b. Persona perception scale: development and exploratory validation of an instrument for evaluating individuals’ perceptions of personas. International Journal of Human-Computer Studies, 141:102437. find
  51. Indira Sen, Marlene Lutz, Elisa Rogers, David Garcia, and Markus Strohmaier. 2025. Missing the margins: A systematic literature review on the demographic representativeness of LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 24263–24289, Vienna, Austria. Associatio… find
  52. Krithi Shailya, Akhilesh Kumar Mishra, Gokul S Krishnan, and Balaraman Ravindran. 2025. Where should I study? Biased language models decide! Evaluating fairness in LMs for academic recommendations. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th C… find
  53. C. Spearman. 1904. The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72–101. find
  54. Ivan Srba, Branislav Pecher, Matus Tomlein, Robert Moro, Elena Stefancova, Jakub Simko, and Maria Bielikova. 2022. Monant medical misinformation dataset: Mapping articles to fact-checked claims. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information… find
  55. Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, and Deep Ganguli. 2023. Evaluating and mitigating discrimination in language model decisions. Preprint, arXiv:2312.03689. find
  56. Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. 2024. Democratizing large language models via personalized parameter-efficient fine-tuning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6476–6491, Miami, Florida, USA.…
  57. Manuel Tonneau, Neil K. R. Seghal, Niyati Malhotra, Sharif Kazemi, Victor Orozco-Olvera, Ana María Muñoz Boudet, Lakshmi Subramanian, Samuel P. Fraiberger, Sharath Chandra Guntuku, and Valentin Hofmann. 2026. Different demographic cues yield inconsistent conclusions about LLM personalization and bia… — [arXiv:2601.18486](https://arxiv.org/abs/2601.18486) find
  58. Kimberly Truong, Riccardo Fogliato, Hoda Heidari, and Steven Wu. 2025. Persona-augmented benchmarking: Evaluating LLMs across diverse writing styles. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 22687–22720, Suzhou, China. Association for Computati… find
  59. John W Tukey. 1949. Comparing individual means in the analysis of variance. Biometrics, pages 99–114. find
  60. Pranav Narayanan Venkit, Jiayi Li, Yingfan Zhou, Sarah Rajtmajer, and Shomir Wilson. 2026a. A tale of two identities: An ethical audit of ai-crafted synthetic personas. Proceedings of the AAAI Conference on Artificial Intelligence, 40(44):37765–37774. find
  61. Pranav Narayanan Venkit, Yu Li, Yada Pruksachatkun, and Chien-Sheng Wu. 2026b. The need for a socially-grounded persona framework for user simulation. Preprint, arXiv:2601.07110. — [arXiv:2601.07110](https://arxiv.org/abs/2601.07110) find
  62. Anvesh Rao Vijjini, Somnath Basu Roy Chowdhury, and Snigdha Chaturvedi. 2025. Exploring safety-utility trade-offs in personalized language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologie… find
  63. Angelina Wang, Daniel E. Ho, and Sanmi Koyejo. 2025. The inadequacy of offline large language model evaluations: A need to account for personalization in model behavior. Patterns, 6(12):101397. find
  64. Kaixun Yang, Mladen Raković, Dragan Gašević, and Guanliang Chen. 2025. Does the prompt-based large language model recognize students’ demographics and introduce bias in essay scoring? In Artificial Intelligence in Education: 26th International Conference, AIED 2025, Palermo, Italy, July 22–26, 202… find
  65. Xiang Zhang, Hans-Frederick Brown, and Anil Shankar. 2016. Data-driven personas: Constructing archetypal users with clickstreams and user telemetry. In Proceedings of the 2016 CHI conference on human factors in computing systems, pages 5350–5359. find
  66. Qishuai Zhong, Zongmin Li, Siqi Fan, and Aixin Sun. 2025. Evaluating LLM adaptation to sociodemographic factors: User profile vs. dialogue history. Preprint, arXiv:2505.21362. — [arXiv:2505.21362](https://arxiv.org/abs/2505.21362) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/weeber-2026-textitone-persona-textitmany-cues-textitdifferent/