ChatHaruhi: Reviving Anime Character in Reality via Large Language Model
Paper · 2023 · Cheng Li, Ziang Leng, Chenxi Yan et al.
Summary
Asking a large language model to simply "act like" a fictional character works only partly. The model relies on its own fuzzy memory of the work, invents facts, and drifts back to its default voice. Fine-tuning on a character's lines tends to increase hallucination and needs more data than minor characters have. ChatHaruhi instead treats original scripts as an external memory. Dialogue exchanges involving the character are extracted from novels, TV shows and films. For each user query, sentence embeddings retrieve the most relevant classic exchanges, which go into the context together with a refined system prompt and the conversation history. The same setup generates extra in-character dialogues, producing ChatHaruhi-54K: 32 Chinese and English TV/anime characters. ChatGLM2-6B is fine-tuned on this data so that one local model can play many roles. Qualitative examples suggest that retrieval plus the improved prompt brings both ChatGPT and ChatGLM2 closer to each character's style. The abstract claims automatic and human evaluation gains, but this v1 report postpones those results. The open data and code give a practical baseline for persona chatbots.
Key points
- ChatHaruhi-54K contains 54,726 dialogues for 32 characters. About 22,752 come from original scripts and over 31,000 were simulated with GPT-3/GPT-4 (the figure also mentions Alpaca-like models). Roughly 15K dialogues are in English and the rest in Chinese.
- The model input follows an s-R-D-H-q layout: system prompt, retrieved character memories D(q, R) from the memory bank U, dialogue history H, and the user query. The reply a is the training target.
- ChatGLM2-6B was fine-tuned for 3 epochs on 4 A100 GPUs. model-A uses only the original dialogues and model-B uses the full 54K set; both are released. model-C, trained on raw character utterances, is planned.
- The qualitative comparison covers five setups: GPT Turbo 3.5 and ChatGLM2 with the system prompt alone, the same two with the full prompt, and fine-tuned ChatGLM2 with the full prompt. Quantitative results and the user study are marked as still in progress.
- Code, data and Hugging Face demos are available at github.com/LC1332/Chat-Haruhi-Suzumiya. The project is part of the Luotuo open-source community and started in a DataWhale study group.
📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works
🧭 Topics 1 Language models & pretraining · 1,343 works
Cite
Cite as Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu and Haozhen Sun. 2023. ChatHaruhi: Reviving Anime Character in Reality via Large Language Model. arXiv:2308.09597.
@misc{li2023chatharuhi,
title = {{ChatHaruhi: Reviving Anime Character in Reality via Large Language Model}},
author = {Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu and Haozhen Sun},
year = {2023},
doi = {10.48550/arxiv.2308.09597},
eprint = {2308.09597},
archivePrefix = {arXiv},
url = {https://doi.org/10.48550/arxiv.2308.09597}
}References
- AIPRM. 2023. Complete small business tasks with prompt management & 1-click prompts for ChatGPT. https://www.aiprm.com/. Accessed: 2023-02-18. ↗
- Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. 2022. Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005. our card
- Anthropic. 2023. Introducing claude. https://www.anthropic.com/index/introducing-claude. Accessed: 2023-2-18. ↗
- OpenAI. 2022. Introducing chatgpt. Open AI Blog. find
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. our card
- Character.AI. 2023. Character ai. https://beta.character.ai/help. Accessed: 2023-08-18. ↗
- Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, and Zornitsa Kozareva. 2022. Improving in-context few-shot learning via self-supervised training. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Lingui… find
- Avia Efrat, Uri Shaham, Dan Kilman, and Omer Levy. 2021. Cryptonite: A cryptic crossword benchmark for extreme ambiguity in language. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4186–4192, Online and Punta Cana, Dominican Republic. Association for… find
- Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021. Surface form competition: Why the highest probability answer isn’t always right. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7038–7051, Online and Punta Cana, Dominic… our card
- Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for "mind" exploration of large scale language model society. find
- OpenAI. 2023. Chatgpt: Optimizing language models for dialogue. find
- Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. Learning to retrieve prompts for in-context learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2655–2671, Seattle, United States. As… our card
- Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Da… our card
- Zheng Liu Shitao Xiao. 2023. Baai general embedding. https://github.com/FlagOpen/FlagEmbedding. ↗
- Liu Siyi, Leng Ziang, Huang Hongsen, Chen Shunian, Hu Jing, Sun Ao, Chen Qiyuan, and Cheng Li. 2023. Luotuo embedding: Generative text embedding model distilled from openai api. https://github.com/LC1332/Luotuo-Text-Embedding. ↗
- Hongjin Su, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, Tao Yu, et al. 2022. One embedder, any task: Instruction-finetuned text embeddings. arXiv preprint arXiv:2212.09741. our card
- Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca. ↗
- He sicheng Wang Yuxin, Sun Qingxuan. 2023. M3e: Moka massive mixed embedding model. find
- Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned language models are zero-shot learners. find
- Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. our card
- Zuobai Zhang. 2020. GitHub - Oxer11/Crosstalk-Generation: Perform crosstalk with Qian Yu — github.com. find
- Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. our card
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2023-chatharuhi-reviving-anime-character-reality/