Elamur Research⌕
☰
home / papers / 2026 / Too Good to be Bad: On the Failure of LLMs to Role-Play Villains

Too Good to be Bad: On the Failure of LLMs to Role-Play Villains

Paper · 2026 · Zihao Yi, Qingxuan Jiang, Ruotian Ma et al.

Summary

LLMs are increasingly used for interactive fiction, games and co-written stories, yet nobody had systematically checked whether they can convincingly play antagonists. The authors hypothesize that safety alignment pushes models toward prosocial, honest behavior, and that this conflicts with voicing selfish or malicious characters, even in clearly fictional settings. To test this, they build the Moral RolePlay benchmark from the COSER corpus of character-centred scenes. The scenes are cleaned and then annotated by gemini-2.5-pro for:

  • context completeness,
  • emotional tone,
  • personality traits,
  • a four-tier moral scale running from paragons through flawed heroes and egoists to villains.

Leading LLMs are evaluated zero-shot on a test set balanced across the four tiers, using prompts that cast the model as an actor. Role-play fidelity drops steadily as character morality decreases, with the largest drop between well-meaning flawed characters and self-serving ones. Models perform worst on deception and manipulation, and they tend to replace subtle scheming with crude hostility. A model's general chatbot ranking says little about how well it plays villains. The work puts numbers on a real tension between safety and creative authenticity.

Key points

  • The full Moral RolePlay dataset has 23,191 scenes and 54,591 character portrayals. Villains (Level 4) make up only 2.6% of it, so the test set uses stratified sampling: 800 characters from 325 scenes, with 200 per moral level.
  • Characters are described with 77 traits, each grouped as positive, neutral or negative. Scenes with a completeness score below 3 were removed, and the remaining scenes average 4.22.
  • Mean fidelity decreases monotonically across the moral tiers: paragons average 3.21, while villains average 2.62. The steepest drop is between Level 2 and Level 3.
  • Models lose the most points on "Manipulative", "Deceitful" and "Cruel". Strongly safety-aligned models degrade disproportionately on villain roles.
  • The authors release a Villain RolePlay (VRP) leaderboard showing that Arena-style chatbot rankings do not match villain role-play quality. The remaining 50k+ annotated portrayals are offered as a training resource.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢TencentShenzhen · China134 works · 19,743 citations🎓Sun Yat-sen UniversityGuangzhou · China53 works · 9,185 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Zihao Yi, Qingxuan Jiang, Ruotian Ma, Xingyu Chen, Qu Yang, Mengru Wang, Fanghua Ye, Ying Shen, Zhaopeng Tu, Xiaolong Li and Liefeng Bo. 2026. Too Good to be Bad: On the Failure of LLMs to Role-Play Villains. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.282

@inproceedings{yi2026too,
  title = {{Too Good to be Bad: On the Failure of LLMs to Role-Play Villains}},
  author = {Zihao Yi, Qingxuan Jiang, Ruotian Ma, Xingyu Chen, Qu Yang, Mengru Wang, Fanghua Ye, Ying Shen, Zhaopeng Tu, Xiaolong Li and Liefeng Bo},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.282},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.282}
}

References

31 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2025. System card: Claude opus 4 & claude sonnet 4. https://www-cdn.anthropic.com. ↗
  2. Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Gao Xing, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, and Fei Huang. 2024a. Socialbench: Sociality evaluation of role-playing conversational agents. In Findings of the Association for Computational Linguistics ACL 2024, pages 2108– 2126. find
  3. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  4. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2023. Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXiv:2310.12773.
  5. Chia-Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. 2024. Safe lora: The silver lining of reducing safety risks when fine-tuning large language models. Advances in Neural Information Processing Systems, 37:65072–65094. find
  6. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  7. Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Vinayak Bhalerao, Christopher Buckley, Jason Phang, Samuel R Bowman, and Ethan Perez. 2023. Pretraining language models with human preferences. In International Conference on Machine Learning, pages 17506–17533. PMLR.
  8. Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao Guo, Da Chen, Dong Li, and 1 others. 2025. Minimax-01: Scaling foundation models with lightning attention. arXiv preprint arXiv:2501.08313.
  9. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  10. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22. find
  11. Yiting Ran, Xintao Wang, Tian Qiu, Jiaqing Liang, Yanghua Xiao, and Deqing Yang. 2025. Bookworld: From novels to interactive agent societies for creative story generation. arXiv preprint arXiv:2504.14538. — [arXiv:2504.14538](https://arxiv.org/abs/2504.14538) find
  12. Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, and 1 others. 2024b. From persona to personalization: A survey on role-playing language agents. arXiv preprint arXiv:2404.18231.
  13. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025a. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.20534.
  14. Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. 2025. Persona vectors: Monitoring and controlling character traits in language models. arXiv preprint arXiv:2507.21509. — [arXiv:2507.21509](https://arxiv.org/abs/2507.21509) find
  15. Tencent Hunyuan Team, Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, ChenChen Zhang, Chengcheng Xu, Chenhao Wang, Decheng Wu, Dengpeng Wu, and 1 others. 2025b. Hunyuan-turbos: Advancing large language models through mamba-transformer synergy and adaptive chain-of-thought. arXiv preprint arXiv:2505.15431.
  16. Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. 2024. Charactereval: A chinese benchmark for role-playing conversational agent evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1183…
  17. Xintao Wang, Heng Wang, Yifei Zhang, Xinfeng Yuan, Rui Xu, Jen-tse Huang, Siyu Yuan, Haoran Guo, Jiangjie Chen, Shuchang Zhou, and 1 others. 2025a. Coser: Coordinating llm-based persona simulation of established roles. arXiv preprint arXiv:2502.09082. — [arXiv:2502.09082](https://arxiv.org/abs/2502.09082) find
  18. Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, and 1 others. 2024. Incharacter: Evaluating personality fidelity in role-playing agents through psychological interviews. In Proceedings of the 62nd Annual Meeting of the Association for…
  19. Zongsheng Wang, Kaili Sun, Bowen Wu, Qun Yu, Ying Li, and Baoxun Wang. 2025b. Raiden-r1: Improving role-awareness of llms via grpo with verifiable reward. arXiv preprint arXiv:2505.10218. — [arXiv:2505.10218](https://arxiv.org/abs/2505.10218) find
  20. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Jiahao Xu, Tian Liang, Pinjia He, and Zhaopeng Tu. 2025. Refuse whenever you feel unsafe: Improving safety in LLMs via decoupled refusal training. In ACL.
  21. Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and 1 others. 2025. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471.
  22. Bang Zhang, Ruotian Ma, Qingxuan Jiang, Peisong Wang, Jiaqi Chen, Zheng Xie, Xingyu Chen, Yue Wang, Fanghua Ye, Jian Li, Yifan Yang, Zhaopeng Tu, and Xiaolong Li. 2025. Sentient agent as a judge: Evaluating higher-order social cognition in large language models. Preprint, arXiv:2505.02847.
  23. Jinfeng Zhou, Yongkang Huang, Bosi Wen, Guanqun Bi, Yuxuan Chen, Pei Ke, Zhuang Chen, Xiyao Xiao, Libiao Peng, Kuntian Tang, and 1 others. 2025. Characterbench: Benchmarking character customization of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39,… find
  24. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593. find
  25. Bingbing Wen, Jihan Yao, Shangbin Feng, Chenjun Xu, Yulia Tsvetkov, Bill Howe, and Lucy Lu Wang. 2025. Know your limits: A survey of abstention in large language models. Transactions of the Association for Computational Linguistics, 13:529–556. find
  26. Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, and Yanghua Xiao. 2024. Character is destiny: Can large language models simulate persona-driven decisions in role-playing? CoRR. find
  27. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  28. Zihao Yi, Jiarui Ouyang, Zhe Xu, Yuwen Liu, Tianhao Liao, Haohao Luo, and Ying Shen. 2024. A survey on recent advances in llm-based multi-turn dialogue systems. ACM Computing Surveys.
  29. Pengfei Yu, Dongming Shen, Silin Meng, Jaewon Lee, Weisu Yin, Andrea Yaoyun Cui, Zhenlin Xu, Yi Zhu, Xingjian Shi, Mu Li, and 1 others. 2025. Rpgbench: Evaluating large language models as role-playing game engines. arXiv preprint arXiv:2502.00595. — [arXiv:2502.00595](https://arxiv.org/abs/2502.00595) find
  30. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher. In ICLR. *Figure 3: Performance of third-person (default) and first-person roleplay.* *Table 8: Impact of reasoning on role-pla…
  31. Level 2 Level 3 Level 4 gemini-2.5-pro × ✓ 3.39 3.42 3.43 3.27 2.93 2.97 2.80 2.75 claude-opus-4.1 × ✓ 3.23 3.20 3.08 3.02 2.63 2.54 2.48 2.53 claude-sonnet-4.5 × ✓ 3.20 3.35 3.37 3.23 2.89 2.83 2.56 2.54 qwen3-max × ✓ 3.19 2.83 2.80 2.83 2.53 2.18 2.48 2.33 grok-4-fast × ✓ 3.15 3.25 2.98 3.09 2.53… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yi-2026-too-good-be-bad-failure/