Elamur Research⌕
☰
home / papers / 2026 / Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs

Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs

Paper · 2026 · Kai Wang, Haoyang You, Yang Zhang et al.

Summary

LLMs used for role-playing often fail to stay in character over long, open-ended conversations. They tend not to recall or correctly apply their assigned persona knowledge unless the scene is spelled out for them. The authors draw on Stanislavski's idea of emotional memory and define Memory-Driven Role-Playing (MDRP). In this task, the persona is stored as long-term memory, the recent dialogue turns act as short-term memory, and the model must find the relevant persona facets using only the conversation itself.

They build three components:

  • MREval, a diagnostic framework that splits role-playing into four memory abilities, each scored by two metrics.
  • MRPrompt, a prompt-only method that restructures persona descriptions into a hierarchical schema and steers retrieval through a "Magic-If" protocol.
  • MRBench, a Chinese/English benchmark built from novels.

Across 12 LLMs, MRPrompt raises a small open model to the level of much larger closed systems. Stronger upstream memory abilities also tend to go with better final responses. This suggests that making memory explicit, rather than scaling the model, is a cheap way to get more faithful personas.

Key points

  • MREval's four abilities are Anchoring, Selecting, Bounding and Enacting (MA/MS/MB/ME). They yield eight metrics, each rated on a 1–10 Likert-style scale.
  • MRBench is bilingual and draws on 10 English and 6 Chinese novels. It was used to evaluate 12 LLMs.
  • Qwen3-8B with MRPrompt averages 8.12. This edges out the base GLM-4.7 (8.11) and base Qwen3-Max (8.08), and comes close to the best result, GLM-4.7 with MRPrompt (8.13).
  • The judge is GPT-4.1-mini, checked against human ratings on 800 instances (100 per metric, split evenly EN/ZH). Judge scores are calibrated per metric, and robustness was additionally tested with Qwen3-32B and four human raters.
  • The combined upstream score (MA/MS/MB) correlates moderately with ME (r=0.35). Limitations include:
    • a next-turn-only evaluation setting;
    • personas drawn only from literary characters;
    • reliance on a single main judge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harbin Institute of TechnologyHarbin · China158 works · 28,841 citations🎓Macquarie UniversitySydney · Australia42 works · 5,776 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Kai Wang, Haoyang You, Yang Zhang and Zhongjie Wang. 2026. Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1175

@inproceedings{wang2026memory,
  title = {{Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs}},
  author = {Kai Wang, Haoyang You, Yang Zhang and Zhongjie Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1175},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1175}
}

References

55 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. AI@Meta. 2024a. Llama 3 model card. Accessed: 2026-01-05. find
  2. AI@Meta. 2024b. Llama 3.2: Model cards and prompt formats. Accessed: 2026-01-05. find
  3. Alan Baddeley. 1992. Working memory. Science, 255(5044):556–559. find
  4. Wenrui Bao, Kai Wang, Siqiang Luo, and Xiang Li. 2025. Permitted knowledge boundary: Evaluating the knowledge-constrained responsiveness of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 13390–13405, Suzhou, China. Association for Computational… find
  5. Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, and 81 others. 2024. InternLM2 Technical Report. Preprint, arXiv:2403.17297.
  6. Mingyu Chen, Jingkai Lin, Zhaojie Chu, Xiaofen Xing, Yirong Chen, and Xiangmin Xu. 2025a. CATCH: A novel data synthesis framework for high therapy fidelity and memory-driven planning chain of thought in AI counseling. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 10… find
  7. Weishu Chen, Jinyi Tang, Zhouhui Hou, Shihao Han, Mingjie Zhan, Zhiyuan Huang, Delong Liu, Jiawei Guo, Zhicheng Zhao, and Fei Su. 2025b. MOOM: Maintenance, organization and optimization of memory in ultra-long role-playing dialogues. Preprint, arXiv:2509.11860. — [arXiv:2509.11860](https://arxiv.org/abs/2509.11860) find
  8. DeepSeek. 2025. Models & pricing (deepseek api docs). Accessed: 2026-01-05. find
  9. Yifan Duan, Yihong Tang, Kehai Chen, Liqiang Nie, and Min Zhang. 2025. ORPP: Self-optimizing role-playing prompts to enhance language model capabilities. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28573–28588, Suzhou, China. Association for Compu… find
  10. Yassine El Boudouri, Walter Nuninger, Julian Alvarez, and Yvan Peter. 2025. Role-playing evaluation for Large Language Models. In Methodologies and Intelligent Systems for Technology Enhanced Learning, 15th International Conference (MIS4TEL 2025), volume 1619 of Lecture Notes in Networks and Systems… find
  11. Feiteng Fang, Ting-En Lin, Yuchuan Wu, Xiong Liu, Xiang Huang, Dingwei Chen, Jing Ye, Haonan Zhang, Liang Zhu, Hamid Alinejad-Rokny, Min Yang, Fei Huang, and Yongbin Li. 2025. ChARM: Character-based act-adaptive reward modeling for advanced role-playing language agents. Preprint, arXiv:2505.23923. — [arXiv:2505.23923](https://arxiv.org/abs/2505.23923) find
  12. Bangkok, Thailand. Association for Computational Linguistics. find
  13. Xiaoxu Han, Wanqing Zhao, Ziyu Guan, and Jinye Peng. 2026. Act-LLM: A whole-process chain for character-centric role-playing with large language models. Expert Systems with Applications, 296:129024. find
  14. Minsik Oh, Joosung Lee, Jiwei Li, and Guoyin Wang. 2024. PK-ICR: Persona-knowledge interactive context retrieval for grounded dialogue. Preprint, arXiv:2302.06674. — [arXiv:2302.06674](https://arxiv.org/abs/2302.06674) find
  15. Kai He, Yucheng Huang, Wenqing Wang, Delong Ran, Dongming Sheng, Junxuan Huang, Qika Lin, Jiaxing Xu, Wenqiang Liu, and Mengling Feng. 2025. Crab: A novel configurable role-playing LLM with assessing benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistic… find
  16. Le Huang, Hengzhi Lan, Zijun Sun, Chuan Shi, and Ting Bai. 2024. Emotional RAG: Enhancing role-playing agents through emotional retrieval. In 2024 IEEE International Conference on Knowledge Graph (ICKG), pages 120–127. IEEE. find
  17. Yoonna Jang, Jungwoo Lim, Yuna Hur, Dongsuk Oh, Suhyune Son, Yeonsoo Lee, Donghoon Shin, Seungryong Kim, and Heuiseok Lim. 2022. Call for customized conversation: Customized conversation grounding persona and knowledge. Preprint, arXiv:2112.08619. — [arXiv:2112.08619](https://arxiv.org/abs/2112.08619) find
  18. Ke Ji, Yixin Lian, Linxu Li, Jingsheng Gao, Weiyuan Li, and Bin Dai. 2025. Enhancing persona consistency for LLMs’ role-playing using persona-aware contrastive learning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26221–26238, Vienna, Austria. Association for Comput… find
  19. Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi Mi, Yaying Fei, Xiaoyang Feng, Song Yan, Haosheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun. 2023. ChatHaruhi: Reviving anime character in reality via large language model. Preprint, arXiv:2308.09597.
  20. Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of Psychology, 22(140):1–55. find
  21. Jiaheng Liu, Zehao Ni, Haoran Que, Tao Sun, Zekun Wang, Jian Yang, Jiakai Wang, Hongcheng Guo, Zhongyuan Peng, Ge Zhang, Jiayi Tian, Xingyuan Bu, Ke Xu, Wenge Rong, Junran Peng, and Zhaoxiang Zhang. 2024. RoleAgent: Building, interacting, and benchmarking high-quality role-playing agents from script… find
  22. Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. 2024. Large language models are superpositions of all characters: Attaining arbitrary role-play via self-alignment. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7828–7840,…
  23. Zhiyang Qi, Takumasa Kaneko, Keiko Takamizo, Mariko Ukiyo, and Michimasa Inaba. 2025. KokoroChat: A Japanese Psychological Counseling Dialogue Dataset Collected via Role-Playing by Trained Counselors. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume… find
  24. Qwen Team. 2025a. Qwen3-Max: Just Scale it. Accessed: 2026-01-05. find
  25. Qwen Team. 2025b. Qwen3 technical report. Preprint, arXiv:2505.09388.
  26. Yiting Ran, Xintao Wang, Tian Qiu, Jiaqing Liang, Yanghua Xiao, and Deqing Yang. 2025. BOOK-WORLD: From novels to interactive agent societies for story creation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15898– 15912, Vi… find
  27. Saksorn Ruangtanusak, Pittawat Taveekitworachai, and Kunat Pipatanakul. 2025. Talk less, call right: Enhancing role-play LLM agents with automatic prompt optimization and role prompting. Preprint, arXiv:2509.00482. — [arXiv:2509.00482](https://arxiv.org/abs/2509.00482) find
  28. Yunfan Shao, Linyang Li, Junqi Dai, and Xipeng Qiu. 2023. Character-LLM: A trainable agent for role-playing. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13153–13187, Singapore. Association for Computational Linguistics.
  29. Jisu Shin, Juhyun Oh, Eunsu Kim, Hoyun Song, and Alice Oh. 2025. Spotting out-of-character behavior: Atomic-level evaluation of persona fidelity in open-ended generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 26312–26332, Vienna, Austria. Association for Compu… find
  30. Constantin Stanislavski. 1989. An actor prepares. Routledge. find
  31. Yihong Tang, Kehai Chen, Muyun Yang, Zhengyu Niu, Jing Li, Tiejun Zhao, and Min Zhang. 2025a. Thinking in character: Advancing role-playing agents with role-aware reasoning. Preprint, arXiv:2506.01748. — [arXiv:2506.01748](https://arxiv.org/abs/2506.01748) find
  32. Yihong Tang, Bo Wang, Xu Wang, Dongming Zhao, Jing Liu, Ruifang He, and Yuexian Hou. 2025b. Role-Break: Character hallucination as a jailbreak attack in role-playing systems. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7386–7402, Abu Dhabi, UAE. Associatio… find
  33. Jiangjie Chen, Shuchang Zhou, Wei Wang, and Yanghua Xiao. 2025b. CoSER: Coordinating LLM-based persona simulation of established roles. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 64822–64858. PMLR. find
  34. Team GLM. 2024. ChatGLM: A family of large language models from glm-130b to glm-4 all tools. Preprint, arXiv:2406.12793.
  35. Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. 2024c. InCharacter: Evaluating personality fidelity in role-playing agents through psychological interviews. In Proceedings of the 62nd Annual…
  36. Team GLM. 2025. GLM-4.7 Overview (Z.AI Developer Docs). Accessed: 2026-01-05. find
  37. Quan Tu, Chuanqi Chen, Jinpeng Li, Yanran Li, Shuo Shang, Dongyan Zhao, Ran Wang, and Rui Yan. 2023. CharacterChat: Learning towards conversational AI with personalized social support. Preprint, arXiv:2308.10278. — [arXiv:2308.10278](https://arxiv.org/abs/2308.10278) find
  38. Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. 2024. CharacterEval: A Chinese benchmark for role-playing conversational agent evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1183…
  39. Endel Tulving and Donald M. Thomson. 1973. Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80(5):352–373. find
  40. Volcengine. 2025. doubao-seed-1.6 (volcengine ark model documentation). Accessed: 2026-01-05. find
  41. Kai Wang, Yuwei Xu, Zhiyong Wu, and Siqiang Luo. 2024a. LLM as Prompter: Low-resource inductive reasoning on arbitrary knowledge graphs. Preprint, arXiv:2402.11804. find
  42. Lei Wang, Jianxun Lian, Yi Huang, Yanqi Dai, Haoxuan Li, Xu Chen, Xing Xie, and Ji-Rong Wen. 2025a. CharacterBox: Evaluating the role-playing capabilities of LLMs in text-based virtual worlds. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computa… find
  43. Noah Wang, Z.y. Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, Man Zhang, Zhaoxiang Zhang, Wanli Ouyang, Ke Xu, Wenhao Huang, Jie Fu, and Junran Peng. 2024b. RoleLLM: Benchmarking, eliciting, and enhancing role-playing abilities of large l…
  44. Xintao Wang, Heng Wang, Yifei Zhang, Xinfeng Yuan, Rui Xu, Jen-Tse Huang, Siyu Yuan, Haoran Guo, Yongjie Wang, Jonathan Leung, and Zhiqi Shen. 2025c. RoleRAG: Enhancing LLM role-playing via graph guided retrieval. Preprint, arXiv:2505.18541. — [arXiv:2505.18541](https://arxiv.org/abs/2505.18541) find
  45. Shihao Yang, Zhicong Lu, Yong Yang, Bo Lv, Yang Shen, and Nayu Liu. 2025. HyCoRA: Hyper-contrastive role-adaptive learning for role-playing. Preprint, arXiv:2511.08017. — [arXiv:2511.08017](https://arxiv.org/abs/2511.08017) find
  46. Bingsheng Yao, Bo Sun, Yuanzhe Dong, Yuxuan Lu, and Dakuo Wang. 2025. DPRF: A generalizable dynamic persona refinement framework for optimizing behavior alignment between personalized LLM role-playing agents and humans. Preprint, arXiv:2510.14205. — [arXiv:2510.14205](https://arxiv.org/abs/2510.14205) find
  47. Pengfei Yu, Dongming Shen, Silin Meng, Jaewon Lee, Weisu Yin, Andrea Yaoyun Cui, Zhenlin Xu, Yi Zhu, Xingjian Shi, Mu Li, and Alex Smola. 2025a. RPG-Bench: Evaluating large language models as role-playing game engines. Preprint, arXiv:2502.00595. Also presented at the NeurIPS 2025 Workshop on Scalin… — [arXiv:2502.00595](https://arxiv.org/abs/2502.00595) find
  48. Xiaoyan Yu, Tongxu Luo, Yifan Wei, Fangyu Lei, Yiming Huang, Hao Peng, and Liehuang Zhu. 2024. Neeko: Leveraging dynamic LoRA for efficient multi-character role-playing agent. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12540–12557, Miami, Florida… find
  49. Yeyong Yu, Runsheng Yu, Haojie Wei, Zhanqiu Zhang, and Quan Qian. 2025b. Beyond dialogue: A profile-dialogue alignment framework towards general role-playing language model. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1199…
  50. Chao Zhang, Jianwen Sun, Jie Ma, Yi Yang, and Yawei Luo. 2025a. TeenEmpath: Towards adolescent psychological counseling with multiple personas and strategies. IEEE Transactions on Affective Computing, pages 1–13. find
  51. Pinyi Zhang, Siyu An, Lingfeng Qiao, Yifei Yu, Jingyang Chen, Jie Wang, Di Yin, Xing Sun, and Kai Zhang. 2025b. RolePlot: A systematic framework for evaluating and enhancing the plot-progression capabilities of role-playing agents. In Proceedings of the 63rd Annual Meeting of the Association for Com… find
  52. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbou…
  53. Weixiang Zhao, Yulin Hu, Yang Deng, Jiahe Guo, Xingyu Sui, Xinyang Han, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, and Ting Liu. 2025. Beware of your po! measuring and mitigating AI safety risks in role-play fine-tuning of LLMs. In Proceedings of the 63rd Annual Meeting of the Association for C…
  54. Jinfeng Zhou, Yongkang Huang, Bosi Wen, Guanqun Bi, Yuxuan Chen, Pei Ke, Zhuang Chen, Xiyao Xiao, Libiao Peng, Kuntian Tang, Rongsheng Zhang, Le Zhang, Tangjie Lv, Zhipeng Hu, Hongning Wang, and Minlie Huang. 2025a. CharacterBench: Benchmarking character customization of large language models. In Pr… find
  55. Lingfeng Zhou, Jialing Zhang, Jin Gao, Mohan Jiang, and Dequan Wang. 2025b. PersonaEval: Are LLM evaluators human enough to judge role-play? Preprint, arXiv:2508.10014. Also appears in COLM 2025. — [arXiv:2508.10014](https://arxiv.org/abs/2508.10014) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-memory-driven-role-playing-evaluation/