Elamur Research
☰
home / papers / 2023 / Character-LLM: A Trainable Agent for Role-Playing

Character-LLM: A Trainable Agent for Role-Playing

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2023.emnlp-main.814.pdf

Summary

The paper asks whether an LLM can portray a specific person, including their life history, temperament and feelings, rather than just generic human routines produced by prompting an API such as ChatGPT. The authors propose Character-LLM, which builds role-playing agents through fine-tuning instead of prompting. Their pipeline begins with Experience Reconstruction. Biographical profiles are gathered from Wikipedia. An instruction-following LLM then turns them into short scene outlines and expands those into detailed dialogues and inner thoughts. In the Experience Upload step, an open-source base model (e.g., LLaMA 7B) is trained on these scenes with supervised fine-tuning. A small set of "protective" scenes teaches the agent to plead ignorance about things its character could not know. The agents were assessed through interviews scored by LLM judges and compared against instruction-tuned models such as Alpaca and Vicuna. They recalled their characters' experiences more faithfully, and the protective scenes reduced out-of-character hallucinations. The work matters as a reproducible, trainable alternative to prompt-based personas for NPCs, social-science simulation and interactive services.

Key points

  • Queen Cleopatra, Julius Caesar and Ludwig van Beethoven are among the example characters. All profiles come from public Wikipedia pages, which the authors chose to avoid privacy issues.
  • The data pipeline has three stages: Profile Collection, Scene Extraction and Experience Completion. ChatGPT generates the scene texts, and these are grounded in the collected profile facts.
  • Fewer than 100 protective scenes per character were enough to curb anachronistic answers without harming other abilities. For example, a trained Beethoven declines to write Python code, whereas Alpaca simply complies.
  • The authors list several limitations:
    • There are no standard evaluation protocols, and the study relies on ChatGPT as a judge.
    • Profile-based data cannot capture a whole life.
    • Results depend strongly on the base model.
    • Agents playing flawed or villainous characters could cause harm.
  • Code and datasets are public at github.com/choosewhatulike/trainable-agents.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

169 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 166 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{shao2023character,
  title = {{Character-LLM: A Trainable Agent for Role-Playing}},
  author = {Shao, Yunfan and Li, Linyang and Dai, Junqi and Qiu, Xipeng},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  url = {https://aclanthology.org/2023.emnlp-main.814/}
}

References

35 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. find
  2. Joseph Bates. 1994. The role of emotion in believable agents. Commun. ACM, 37(7):122–125. find
  3. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. find our card
  4. Rodney A. Brooks, Cynthia Breazeal, Marko Marjanovic, Brian Scassellati, and Matthew Williamson. 2000. The cog project: Building a humanoid robot. In Computation for Metaphors, Analogy, and Agents, number 1562 in Lecture Notes on Artificial Intelligence, pages 52–87, Berlin. Springer-Verlag. find
  5. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  6. Chris Callison-Burch, Gaurav Singh Tomar, Lara Martin, Daphne Ippolito, Suma Bailis, and David Reitter. 2022. Dungeons and dragons as a dialog challenge for artificial intelligence. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9379–9393, Abu Dhabi,… find
  7. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. find
  8. Jonas Freiknecht and Wolfgang Effelsberg. 2020. Procedural generation of interactive stories using language models. In International Conference on the Foundations of Digital Games (FDG ’20), page 8, Bugibba, Malta. ACM. find
  9. Zhijiang Guo, Michael Sejr Schlichtkrull, and Andreas Vlachos. 2022. A survey on automated fact-checking. Trans. Assoc. Comput. Linguistics, 10:178–206. find our card
  10. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38. find our card
  11. Hyunwoo Kim, Jack Hessel, Liwei Jiang, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, et al. 2022. Soda: Million-scale dialogue distillation with social commonsense contextualization. arXiv preprint arXiv:2212.10465. find our card
  12. Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 9332– 9346. Ass… find
  13. John Laird and Michael VanLent. 2001. Human-level ai’s killer application: Interactive computer games. AI Magazine, 22(2):15. find
  14. Neil Madden and Brian Logan. 2007. Collaborative narrative generation in persistent virtual environments. In Intelligent Narrative Technologies, Papers from the 2007 AAAI Fall Symposium, Arlington, Virginia, USA, November 9-11, 2007, volume FS-07-05 of AAAI Technical Report, pages 71–78. AAAI Press. find
  15. Shohei Miyashita, Xinyu Lian, Xiao Zeng, Takashi Matsubara, and Kuniaki Uehara. 2017. Developing game ai agent behaving like human by mixing reinforcement learning and supervised learning. In Proceedings of the 18th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence,… find
  16. Thanh Thi Nguyen, Quoc Viet Hung Nguyen, Dung Tien Nguyen, Duc Thanh Nguyen, Thien Huynh-The, Saeid Nahavandi, Thanh Tam Nguyen, Quoc-Viet Pham, and Cuong M Nguyen. 2022. Deep learning for deepfakes creation and detection: A survey. Computer Vision and Image Understanding, 223:103525. find
  17. OpenAI. 2023. GPT-4 technical report. abs/2303.08774. find our card
  18. CoRR, Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744. find
  19. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. find
  20. Joon Sung Park, Lindsay Popowski, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. Social simulacra: Creating populated prototypes for social computing systems. In In the 35th Annual ACM Symposium on User Interface Software and Technology (UIST ’22), UIST ’22, New… find
  21. Mark O. Riedl. 2012. Interactive narrative: A novel application of artificial intelligence for computer games. In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI’12), pages 2160– 2165. find
  22. Mark O. Riedl and R. Michael Young. 2005. An objective character believability evaluation procedure for multi-agent story generation systems. In Proceedings of the 5th International Working Conference on Intelligent Virtual Agents (IVA’05), pages 58–70, Kos, Greece. find
  23. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford\_alpaca. ↗
  24. Frank Thomas and Ollie Johnston. 1981. Disney Animation: The Illusion of Life. Abbeville Press, New York. find
  25. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. find
  26. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023a. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111. find our card
  27. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023b. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. find
  28. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560. find
  29. Yuhan Wang, Xu Chen, Junwei Zhu, Wenqing Chu, Ying Tai, Chengjie Wang, Jilin Li, Yongjian Wu, Feiyue Huang, and Rongrong Ji. 2021. Hififace: 3d shape and semantic prior guided high fidelity face swapping. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJ-C… find
  30. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS. find our card
  31. Tongshuang Wu, Ellen Jiang, Aaron Donsbach, Jeff Gray, Alejandra Molina, Michael Terry, and Carrie J Cai. 2022a. Promptchainer: Chaining large language model prompts through visual programming. In CHI EA ’22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems. find
  32. Tongshuang Wu, Michael Terry, and Carrie J Cai. 2022b. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In CHI ’22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. find
  33. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244. find our card
  34. Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023. Speak foreign languages with your own voice: Cross-lingual neural codec language modeling. arXiv preprint arXiv:2303.03926. — [arXiv:2303.03926](https://arxiv.org/abs/2303.03926) find
  35. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206. find