Elamur Research⌕
☰
home / papers / 2025 / LLMs + Persona-Plug = Personalized LLMs

LLMs + Persona-Plug = Personalized LLMs

Paper · 2025 · Jiongnan Liu, Yutao Zhu, Shuting Wang et al.

Summary

Making LLMs adapt to individual users usually means either training a separate model per user, which is costly and fails when user data is scarce, or retrieving a few relevant past texts as demonstrations. Retrieval tends to miss a person's overall style. The authors propose PPlug (persona-plug), which leaves the LLM frozen and adds a small, shared user embedder. Each past behavior of a user is encoded into a vector by a frozen BGE-base-en-v1.5 encoder. A tunable input encoder and an input-aware personal aggregator then weight these vectors by their relevance to the current request and merge all of them into one personal embedding. This embedding is prepended to the LLM's input. The embedder is trained end-to-end on data from all users, so no per-user parameters are needed. On six tasks of the LaMP benchmark, PPlug beats existing personalized LLM approaches by 1.4% to 35.8%. The work shows that a compact, holistic user representation can personalize outputs cheaply and in a plug-and-play way.

Key points

  • Architecture: a frozen history encoder (BGE-base-en-v1.5) and a fine-tuned input encoder feed an input-aware aggregator. The aggregator outputs a single user embedding for a fixed LLM, adding no parameters to the LLM itself.
  • Main result: on six LaMP tasks, PPlug improves over prior personalized LLMs by 1.4–35.8%.
  • History selection: restricting aggregation to the top-4 weighted histories ("PPlug with Selection"), mimicking retrieval methods, lowers performance. Using all histories works better.
  • Hybrid approach: combining PPlug with retrieved demonstrations ("PPlug + Retrieval") gives further gains. The coarse style embedding and fine-grained retrieved examples complement each other.
  • Effect of history length (validation set):
    • On LaMP-3, RMSE falls from 0.548 to 0.522 between users with short and long histories.
    • On LaMP-5, ROUGE-1 rises from 0.474 to 0.499 across the same groups.
    • The method stays robust for users with little history.
    • The authors note privacy benefits: users could compute their embedding locally instead of uploading their texts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BaiduBeijing · China74 works · 15,263 citations🎓Renmin University of ChinaBeijing · China114 works · 16,118 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Jiongnan Liu, Yutao Zhu, Shuting Wang, Xiaochi Wei, Erxue Min, Yu Lu, Shuaiqiang Wang, Dawei Yin and Zhicheng Dou. 2025. LLMs + Persona-Plug = Personalized LLMs. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.461

@inproceedings{liu2025llms,
  title = {{LLMs + Persona-Plug = Personalized LLMs}},
  author = {Jiongnan Liu, Yutao Zhu, Shuting Wang, Xiaochi Wei, Erxue Min, Yu Lu, Shuaiqiang Wang, Dawei Yin and Zhicheng Dou},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.461},
  url = {https://doi.org/10.18653/v1/2025.acl-long.461}
}

References

41 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Se… — PaLM 2 Technical Report
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  3. Jin Chen, Zheng Liu, Xu Huang, Chenwang Wu, Qi Liu, Gangwei Jiang, Yuanhao Pu, Yuxuan Lei, Xiaolong Chen, Xingmei Wang, Defu Lian, and Enhong Chen. 2023. When large language models meet personalization: Perspectives of challenges and opportunities. CoRR, abs/2307.16376. find
  4. Konstantina Christakopoulou, Alberto Lalama, Cj Adams, Iris Qu, Yifat Amir, Samer Chucri, Pierce Vollucci, Fabio Soldo, Dina Bseiso, Sarah Scodel, Lucas Dixon, Ed H. Chi, and Minmin Chen. 2023. Large language models for user interest journeys. CoRR, abs/2305.15498. — [arXiv:2305.15498](https://arxiv.org/abs/2305.15498) find
  5. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang… find
  6. Sumanth Doddapaneni, Krishna Sayana, Ambarish Jash, Sukhdeep S. Sodhi, and Dima Kuzmin. 2024. User embedding model for personalized language prompting. CoRR, abs/2401.04858. find
  7. Markus Freitag and Yaser Al-Onaizan. 2017. Beam search strategies for neural machine translation. In NMT@ACL, pages 56–60. Association for Computational Linguistics.
  8. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. In EMNLP (1), pages 6894–6910. Association for Computational Linguistics.
  9. Songwei Ge, Zhicheng Dou, Zhengbao Jiang, Jian-Yun Nie, and Ji-Rong Wen. 2018. Personalizing search results using hierarchical RNN with query-aware attention. In CIKM, pages 347–356. ACM. find
  10. Liam Hebert, Krishna Sayana, Ambarish Jash, Alexandros Karatzoglou, Sukhdeep S. Sodhi, Sumanth Doddapaneni, Yanli Cai, and Dima Kuzmin. 2024. PER-SOMA: personalized soft prompt adapter architecture for personalized language prompting. CoRR, abs/2408.00960. — [arXiv:2408.00960](https://arxiv.org/abs/2408.00960) find
  11. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In ICLR. OpenReview.net.
  12. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2021. Towards unsupervised dense information retrieval with contrastive learning. CoRR, abs/2112.09118.
  13. Wang-Cheng Kang and Julian J. McAuley. 2018. Self-attentive sequential recommendation. In ICDM, pages 197–206. IEEE Computer Society. find
  14. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. CoRR, abs/2001.08361.
  15. Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Dernoncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, and Hamed Zamani. 2024. Longlamp: A benchmark for personalized long-form text generation. CoRR, abs/2407.11016. — [arXiv:2407.11016](https://arxiv.org/abs/2407.11016) find
  16. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  17. Jiongnan Liu, Jiajie Jin, Zihan Wang, Jiehan Cheng, Zhicheng Dou, and Ji-Rong Wen. 2023. RETA-LLM: A retrieval-augmented large language model toolkit. CoRR, abs/2306.05212. — [arXiv:2306.05212](https://arxiv.org/abs/2306.05212) find
  18. John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander M. Rush. 2023. Text embeddings reveal (almost) as much as text. In EMNLP, pages 12448–12460. Association for Computational Linguistics.
  19. Lin Ning, Luyang Liu, Jiaxing Wu, Neo Wu, Devora Berlowitz, Sushant Prakash, Bradley Green, Shawn O’Banion, and Jun Xie. 2024. User-llm: Efficient LLM contextualization with user embeddings. CoRR, abs/2402.13598. — [arXiv:2402.13598](https://arxiv.org/abs/2402.13598) find
  20. Christopher Richardson, Yao Zhang, Kellen Gillespie, Sudipta Kar, Arshdeep Singh, Zeynab Raeesy, Omar Zia Khan, and Abhinav Sethy. 2023. Integrating summarization and retrieval for enhanced personalization via large language models. CoRR, abs/2310.20081. — [arXiv:2310.20081](https://arxiv.org/abs/2310.20081) find
  21. Stephen E. Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Found. Trends Inf. Retr., 3(4):333–389. find
  22. Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqi… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  23. Alireza Salemi, Surya Kallumadi, and Hamed Zamani. 2024a. Optimization methods for personalizing large language models through retrieval augmentation. In SIGIR, pages 752–762. ACM. find
  24. Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2024b. Lamp: When large language models meet personalization. In ACL (1), pages 7370–7392. Association for Computational Linguistics.
  25. Alireza Salemi and Hamed Zamani. 2024. Comparing retrieval-augmentation and parameter-efficient fine-tuning for privacy-preserving personalization of large language models. CoRR, abs/2409.09510. — [arXiv:2409.09510](https://arxiv.org/abs/2409.09510) find
  26. Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. One embedder, any task: Instruction-finetuned text embeddings. In ACL (Findings), pages 1102–1121. Association for Computational Linguistics.
  27. Zhaoxuan Tan, Zheyuan Liu, and Meng Jiang. 2024a. Personalized pieces: Efficient personalized large language models through collaborative efforts. In EMNLP, pages 6459–6475. Association for Computational Linguistics.
  28. Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. 2024b. Democratizing large language models via personalized parameter-efficient fine-tuning. In EMNLP, pages 6476–6491. Association for Computational Linguistics.
  29. Xiangru Tang, Xingyao Zhang, Yanjun Shao, Jie Wu, Yilun Zhao, Arman Cohan, Ming Gong, Dongmei Zhang, and Mark Gerstein. 2024. Step-back profiling: Distilling user history for personalized scientific writing. CoRR, abs/2406.14275. find
  30. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. Llama: Open and efficient foundation language models. CoRR, a…
  31. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… find
  32. Stanislaw Wozniak, Bartlomiej Koptyra, Arkadiusz Janz, Przemyslaw Kazienko, and Jan Kocon. 2024. Personalized large language models. CoRR, abs/2402.09269. — [arXiv:2402.09269](https://arxiv.org/abs/2402.09269) find
  33. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. CoRR, abs/2309.07597.
  34. Kai Zhang, Lizhi Qing, Yangyang Kang, and Xiaozhong Liu. 2024a. Personalized LLM response generation with parameterized memory injection. CoRR, abs/2404.03565. — [arXiv:2404.03565](https://arxiv.org/abs/2404.03565) find
  35. Peitian Zhang, Shitao Xiao, Zheng Liu, Zhicheng Dou, and Jian-Yun Nie. 2023. Retrieve anything to augment large language models. CoRR, abs/2310.07554. — [arXiv:2310.07554](https://arxiv.org/abs/2310.07554) find
  36. Zhehao Zhang, Ryan A Rossi, Branislav Kveton, Yijia Shao, Diyi Yang, Hamed Zamani, Franck Dernoncourt, Joe Barrow, Tong Yu, Sungchul Kim, et al. 2024b. Personalization of large language models: A survey. arXiv preprint arXiv:2411.00027.
  37. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of l…
  38. Yutao Zhu, Zhaoheng Huang, Zhicheng Dou, and Ji-Rong Wen. 2025. One token can help! learning scalable and pluggable virtual tokens for retrieval-augmented large language models. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 - March 4, 2025, Phil… find
  39. Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou, and Ji-Rong Wen. 2023. Large language models for information retrieval: A survey. CoRR, abs/2308.07107.
  40. Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zheng Liu, Ji-Rong Wen, and Zhicheng Dou. 2024. INTERS: unlocking the power of large language models in search with instruction tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume… find
  41. Yuchen Zhuang, Haotian Sun, Yue Yu, Rushi Qiang, Qifan Wang, Chao Zhang, and Bo Dai. 2024. HYDRA: model factorization framework for black-box LLM personalization. CoRR, abs/2406.02888. — [arXiv:2406.02888](https://arxiv.org/abs/2406.02888) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2025-llms-persona-plug-personalized-llms/