Elamur Research⌕
☰
home / papers / 2026 / Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations

Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations

Paper · 2026 · Xiaoxu Ma, Xiangbo Zhang and Zhenyu Weng

Summary

Personality tests for LLMs, whether Likert-style self-reports or open-ended answers scored by a judge, are fragile. Small changes in item wording or role-play setup can move the resulting scores noticeably, and the tests reveal little about how a score arises. The authors propose Persona-Vector Neutrality Interpolation (PVNI), which estimates Big Five (OCEAN) traits from hidden-state activations instead of from answers alone.

For each trait, contrasting positive and negative persona prompts produce mean activations, and their difference defines a persona vector. A neutral prompt's representation is then placed along this axis. Its relative position sets an interpolation weight between judge-assigned scores for the two anchor conditions. A linear theory of persona vectors justifies these steps.

On Qwen-2.5-7B, Llama-3-8B and Mistral-7B-v0.1, under both questionnaire and role-play variants, PVNI shows clearly lower variance than standard protocols. It still agrees closely with them. The method therefore offers a more reproducible and geometrically transparent way to profile model behavior, but it needs white-box access.

Key points

  • Baselines compared: open-ended elicitation scored by an LLM judge, and the self-report inventories IPIP-BFFM-50 and IPIP-NEO-120. PVNI gives tighter boxplots, with smaller IQRs and shorter whiskers, across traits and models.
  • Convergent validity is high:
    • Pearson r with open-ended elicitation is 0.948–0.990.
    • Average r with the two self-report inventories is 0.880–0.919.
  • Language ablation on Qwen-2.5-7B: Chinese and English anchor prompts shift the anchor scores, but final PVNI scores barely change (e.g., Openness 83.60 vs. 83.35; Agreeableness 93.16 vs. 93.40).
  • The external judge scores outputs on a 0–100 scale, but only for the anchors. The interpolation weight comes from the model's internal geometry.
  • Limitations:
    • The method requires hidden-state access, so it cannot be applied directly to API-only models.
    • Experiments are mostly single-turn and cover only Big Five traits.
    • The authors frame "personality" as a protocol-conditioned behavioral bias, not a human-like trait.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations🎓South China University of TechnologyGuangzhou · China22 works · 1,554 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksSafety Research · 543 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Xiaoxu Ma, Xiangbo Zhang and Zhenyu Weng. 2026. Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.803

@inproceedings{ma2026stable,
  title = {{Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations}},
  author = {Xiaoxu Ma, Xiangbo Zhang and Zhenyu Weng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.803},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.803}
}

References

41 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024. Llama 3 model card. Model card. find
  2. Nishant Balepur, Rachel Rudinger, and Jordan Lee Boyd-Graber. 2025. Which of these best describes multiple choice evaluation with LLMs? A) forced B) flawed C) fixable D) all of the above. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper…
  3. Pranav Bhandari, Usman Naseem, Amitava Datta, Nicolas Fay, and Mehwish Nasim. 2025. Evaluating personality traits in large language models: Insights from psychological questionnaires. In Companion Proceedings of the ACM on Web Conference 2025, pages 868–872, Sydney NSW, Australia. Association for Co… find
  4. Bojana Bodroža, Bojana M. Dinić, and Ljubiša Bojić. 2024. Personality testing of large language models: limited temporal stability, but highlighted prosociality. Royal Society Open Science, 11(10):240180. find
  5. Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. 2025. Persona vectors: Monitoring and controlling character traits in language models. Computing Research Repository, arXiv:2507.21509. ArXiv preprint. — [arXiv:2507.21509](https://arxiv.org/abs/2507.21509) find
  6. Xi Chen, Wei Xue, and Yike Guo. 2026. ActorMind: Emulating human actor reasoning for speech role-playing. Computing Research Repository, arXiv:2604.11103. ArXiv preprint. find
  7. Adrian de Wynter, Xun Wang, Alex Sokolov, Qilong Gu, and Si-Qing Chen. 2023. An evaluation on large language model outputs: Discourse and memorization. Natural Language Processing Journal, 4:100024. find
  8. Chaoyu Gong, Zhi-Gang Su, Pei-Hong Wang, Qian Wang, and Yang You. 2023. A sparse reconstructive evidential k-nearest neighbor classifier for high-dimensional data. IEEE Transactions on Knowledge and Data Engineering, 35(6):5563–5576. find
  9. Akshat Gupta, Xiaoyang Song, and Gopala Anumanchipalli. 2024. Self-assessment tests are unreliable measures of LLM personality. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 301–314, Miami, Florida, US. Association for Computational Linguis… find
  10. Pengrui Han, Rafal Kocielnik, Peiyang Song, Ramit Debnath, Dean Mobbs, Anima Anandkumar, and R. Michael Alvarez. 2025. The personality illusion: Revealing dissociation between self-reports & behavior in LLMs. Computing Research Repository, arXiv:2509.03730. ArXiv preprint. — [arXiv:2509.03730](https://arxiv.org/abs/2509.03730) find
  11. Zixu Li, Yupeng Hu, Zhiwei Chen, Qinlei Huang, Guozhi Qiu, Zhiheng Fu, and Meng Liu. 2026. Retrack: Evidence-driven dual-stream directional anchor calibration network for composed video retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 23373–23381. find
  12. Robert Hogan, Joyce Hogan, and Brent Roberts. 1996. Personality measurement and employment decisions. American Psychologist, 51:469–477. find
  13. Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A. Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. In Advances in Neural Information Processing Systems, volume 35, pages 1950–1965.
  14. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In Proceedings of the International Conference on Learning Representations.
  15. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  16. Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu. 2023b. Evaluating and inducing personality in pre-trained language models. In Advances in Neural Information Processing Systems, volume 36. find
  17. Yi-Fei Liu, Yi-Long Lu, Di He, and Hang Zhang. 2025. From five dimensions to many: Large language models as precise and interpretable psychological profilers. Computing Research Repository, arXiv:2511.03235. ArXiv preprint. — [arXiv:2511.03235](https://arxiv.org/abs/2511.03235) find
  18. Kexin Ma, Bojun Li, Yuhua Tang, Ruochun Jin, and Liting Sun. 2026. CAST: Character-and-scene episodic memory for agents. Computing Research Repository, arXiv:2602.06051. ArXiv preprint. — [arXiv:2602.06051](https://arxiv.org/abs/2602.06051) find
  19. Xiaoxu Ma, Runhao Li, and Zhenyu Weng. 2025. Mutual learning for hashing: Unlocking strong hash functions from weak supervision. Computing Research Repository, arXiv:2510.07703. ArXiv preprint. — [arXiv:2510.07703](https://arxiv.org/abs/2510.07703) find
  20. Deniz Ones, Chockalingam (Vish) Viswesvaran, and Angelika Reiss. 1996. Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81:660–679. find
  21. Hang Jiang, Xiajie Zhang, Xubo Cao, Cynthia Breazeal, Deb Roy, and Jad Kabbara. 2024. Personallm: Investigating the ability of large language models to express personality traits. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3605–3627, Mexico City, Mexico. Associat… find
  22. Tsung-Min Pai, Jui-I Wang, Li-Chun Lu, Shao-Hua Sun, Hung-yi Lee, and Kai-Wei Chang. 2026. BILLY: Steering large language models via merging persona vectors for creative generation. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume…
  23. Hongkang Li, Yihua Zhang, Shuai Zhang, Pin-Yu Chen, Sijia Liu, and Meng Wang. 2025a. When is task vector provably effective for model editing? a generalization analysis of nonlinear transformers. In International Conference on Learning Representations. find
  24. Karl Pearson. 1896. Mathematical contributions to the theory of evolution.—iii. regression, heredity, and panmixia. Philosophical Transactions of the Royal Society of London. Series A, 187:253–318. find
  25. Junxian Li, Beining Xu, Simin Chen, Jiatong Li, Jingdi Lei, Haodong Zhao, and Di Zhang. 2025b. IAG: Input-aware backdoor attack on VLM-based visual grounding. Computing Research Repository, arXiv:2508.09456. ArXiv preprint. — [arXiv:2508.09456](https://arxiv.org/abs/2508.09456) find
  26. Jivnesh Sandhan, Fei Cheng, Tushar Sandhan, and Yugo Murawaki. 2025. CAPE: Context-aware personality evaluation framework for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 10648–10662, Suzhou, China. Association for Computational Linguistics. find
  27. Wenkai Li, Jiarui Liu, Andy Liu, Xuhui Zhou, Mona T. Diab, and Maarten Sap. 2025c. BIG5-CHAT: Shaping LLM personalities through training on human-grounded data. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 20434– 20471, Vie…
  28. Qwen Team. 2024. Qwen2.5: A party of foundation models! Blog post. find
  29. Gregory Serapio-García, Mustafa Safdari, Clément Crepy, Luning Sun, Stephen Fitz, Peter Romero, Marwa Abdulhai, Aleksandra Faust, and Maja Matarić. 2025. A psychometric framework for evaluating and shaping personality traits in large language models. Nature Machine Intelligence, 7(12):1954– 1968. find
  30. Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens. 2024. You don’t need a personality test to know these models are unreliable: Assessing the reliability of large language models on psychometric instruments. In Proceedings of t… find
  31. Jianfeng Zhu, Julina Maharjan, Xinyu Li, Karin G. Coifman, and Ruoming Jin. 2025a. Evaluating LLM alignment on personality inference from real-world interview data. Computing Research Repository, arXiv:2509.13244. ArXiv preprint. — [arXiv:2509.13244](https://arxiv.org/abs/2509.13244) find
  32. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. 2025b. Personality alignment of large language models. In Proceedings of the International Conference on Learning Representations.
  33. C. Spearman. 1904. The proof and measurement of association between two things. The American Journal of Psychology, 15(1):72–101. find
  34. Seungjong Sun, Seo Yeon Baek, and Jang Hyun Kim. 2025. Personality vector: Modulating personality of large language models by model merging. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 24656– 24677, Suzhou, China. Association for Computational Lin… find
  35. Taiji Suzuki and Atsushi Nitanda. 2021. Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space. In Advances in Neural Information Processing Systems, volume 34, pages 3609–3621. Curran Associates, Inc. find
  36. Yixuan Tang, Yi Yang, and Ahmed Abbasi. 2025. Per-sonafuse: A personality activation-driven framework for enhancing human-LLM interactions. Computing Research Repository, arXiv:2509.07370. ArXiv preprint. — [arXiv:2509.07370](https://arxiv.org/abs/2509.07370) find
  37. Tommaso Tosato, Saskia Helbling, Yorguin-Jose Mantilla-Ramos, Mahmood Hegazy, Alberto Tosato, David John Lemay, Irina Rish, and Guillaume Dumas. 2025. Persistent instability in LLM’s personality measurements: Effects of scale, reasoning, and conversation history. Computing Research Repository, arXiv… — [arXiv:2508.04826](https://arxiv.org/abs/2508.04826) find
  38. Xi Wang, Songlei Jian, Shasha Li, Xiaopeng Li, Zhaoye Li, Bin Ji, Baosheng Wang, and Jie Yu. 2026. JPU: Bridging jailbreak defense and unlearning via on-policy path rectification. Computing Research Repository, arXiv:2601.03005. ArXiv preprint. find
  39. Yilei Wang, Jiabao Zhao, Deniz Ones, Liang He, and Xin Xu. 2025. Evaluating the ability of large language models to emulate personality. Scientific Reports, 15(1):519. find
  40. Yuanjun Zhang, Fuzel Ahamed Shaik, Suvojit Acharjee, Fahad Khalid, and Mourad Oussalah. 2026. Towards reliable multimodal disaster severity assessment through preference optimization and explainable vision-language reasoning. Reliability Engineering & System Safety, page 112674. find
  41. Jingyao Zheng, Xian Wang, Simo Hosio, Xiaoxian Xu, and Lik-Hang Lee. 2025. LMLPA: Language model linguistic personality assessment. Computational Linguistics, 51(2):599–640. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ma-2026-stable-explainable-personality-trait-evaluation/