Elamur Research⌕
☰
home / papers / 2026 / Mirroring Users: Towards Building Preference-aligned User Simulator with User Feedback in Recommendation

Mirroring Users: Towards Building Preference-aligned User Simulator with User Feedback in Recommendation

Paper · 2026 · Tianjun Wei, Huizhong Guo, Yingpeng Du et al.

Summary

Recommender systems need realistic user simulators for offline evaluation, because online A/B tests are slow and raise privacy issues. LLM-based simulators help, but they are rarely tuned to recommendation tasks and are expensive to run at scale. The authors present UserMirrorer, a framework that converts large volumes of raw, noisy recommendation feedback into compact training data. It first rewrites logs from eight domains as unified "simulation scenes," each pairing a user's memory (profile and history) with a list of exposed items. A strong LLM then writes an explicit decision-making rationale for each user choice, which reduces ambiguity. Finally, a distillation step uses uncertainty estimation and behavior sampling to keep the hard, informative, denoised samples that weaker models struggle with. Lightweight LLMs are fine-tuned on this data with SFT and DPO. The resulting simulators match real user choices more closely and give interpretable reasons. Their feedback also improves downstream recommenders when added to training. This makes cheaper, more faithful, open user simulation practical.

Key points

  • The task is a listwise, impression-aware scene: the simulator outputs a categorical distribution over actions on the whole exposure list. For datasets that lack logged exposures, the lists are built with existing recommendation strategies.
  • The data covers eight diverse domains. Examples include MovieLens-style ratings, Steam, and news, converted to plain text through domain-specific templates.
  • Simulator feedback was used to augment 10 RS algorithms, including LightGCN, DiffRec, SASRec and NARM, on MovieLens and Steam. It gave consistent gains in Recall, NDCG and MRR, with relative improvements of roughly 7–20% for the reported models.
  • Released resources include code (github.com/Joinn99/UserMirrorer), a mixed-domain dataset (HF: Joinn/UserMirrorer) and a fine-tuned checkpoint (Joinn/UserMirrorrer-Llama-DPO).
  • Stated limitations: the framework handles no multimodal signals and does not model non-interaction or user-leaving behavior.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Nanyang Technological UniversitySingapore · Singapore156 works · 25,644 citations🎓Zhejiang UniversityHangzhou · China172 works · 12,988 citations🎓Singapore University of Technology and DesignSingapore · Singapore49 works · 11,074 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksInformation Systems · 1,437 worksDecision Sciences · 209 worksManagement Science and Operations Research · 108 works

Cite

Cite as Tianjun Wei, Huizhong Guo, Yingpeng Du, Zhu Sun, Huang Chen, Dongxia Wang and Jie Zhang. 2026. Mirroring Users: Towards Building Preference-aligned User Simulator with User Feedback in Recommendation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.273

@inproceedings{wei2026mirroring,
  title = {{Mirroring Users: Towards Building Preference-aligned User Simulator with User Feedback in Recommendation}},
  author = {Tianjun Wei, Huizhong Guo, Yingpeng Du, Zhu Sun, Huang Chen, Dongxia Wang and Jie Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.273},
  url = {https://doi.org/10.18653/v1/2026.acl-long.273}
}

References

67 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. G. Adomavicius and A. Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6):734–749. find
  2. Thierry Bertin-Mahieux, Daniel P.W. Ellis, Brian Whitman, and Paul Lamere. 2011. The million song dataset. In Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR 2011). find
  3. Shijie Chen, Bernal Jimenez Gutierrez, and Yu Su. 2025. Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers. In The Thirteenth International Conference on Learning Representations. find
  4. Nathan Corecco, Giorgio Piatti, Luca A. Lanzendörfer, Flint Xiaofeng Fan, and Roger Wattenhofer. 2024. SUBER: An RL environment with simulated human behavior for recommender systems. In Proceedings of the 27th European Conference on Artificial Intelligence (ECAI 2024). find
  5. Alex Deng, Jiannan Lu, and Jonthan Litz. 2017. Trustworthy Analysis of Online A/B Tests: Pitfalls, challenges and solutions. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM ’17, pages 641–649, New York, NY, USA. Association for Computing Machinery. find
  6. Jiaxin Deng, Shiyao Wang, Kuo Cai, Lejian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. 2025. OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment. Preprint, arXiv:2502.18965. — [arXiv:2502.18965](https://arxiv.org/abs/2502.18965) find
  7. Mukund Deshpande and George Karypis. 2004. Item-based top-N recommendation algorithms. ACM Trans. Inf. Syst., 22(1):143–177. find
  8. J.F. Engel, R.D. Blackwell, and D.T. Kollat. 1978. Consumer Behavior. Business and Management: Marketing. Dryden Press. find
  9. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proceedings of The 33rd International Conference on Machine Learning, pages 1050–1059. PMLR. find
  10. Chen Gao, Xiaochong Lan, Zhihong Lu, Jinzhu Mao, Jinghua Piao, Huandong Wang, Depeng Jin, and Yong Li. 2023. S3: Social-network Simulation System with Large Language Model-Empowered Agents. Preprint, arXiv:2307.14984. — [arXiv:2307.14984](https://arxiv.org/abs/2307.14984) find
  11. Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. 2022. Kuairec: A fully-observed dataset and insights for evaluating recommender systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management… find
  12. Alexandre Gilotte, Clément Calauzènes, Thomas Ned-elec, Alexandre Abraham, and Simon Dollé. 2018. Offline A/B Testing for Recommender Systems. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM ’18, pages 198–206, New York, NY, USA. Association for Comput… find
  13. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Deepseek-r1 incentivizes reasoning in llms through rein… find
  14. Huizhong Guo, Tianjun Wei, Dongxia Wang, Yingpeng Du, Ziyan Wang, Jie Zhang, and Zhu Sun. 2026. Think when needed: Model-aware reasoning routing for llm-based ranking. Preprint, arXiv:2601.18146. — [arXiv:2601.18146](https://arxiv.org/abs/2601.18146) find
  15. F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1– 19. find
  16. Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, pages 639–… find
  17. Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173–182. find
  18. Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang. 2024. Decomposing uncertainty for large language models through input clarification ensembling. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of ICML’24, pages 19023–19042, Vienna, A… find
  19. Yiqiao Jin, Qinlin Zhao, Yiyang Wang, Hao Chen, Kaijie Zhu, Yijia Xiao, and Jindong Wang. 2024. AgentReview: Exploring Peer Review Dynamics with LLM Agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1208–1226, Miami, Florida, USA. Association fo… find
  20. Daniel Kahneman. 2011. Thinking, Fast and Slow. Thinking, Fast and Slow. Farrar, Straus and Giroux, New York, NY, US. find
  21. Dae Y. Kang, Pamela N. DeYoung, Justin Tantiongloc, Todd P. Coleman, and Robert L. Owens. 2021. Statistical uncertainty quantification to augment clinical decision support: a first implementation in sleep medicine. npj Digital Medicine, 4(1). find
  22. Wang-Cheng Kang and Julian McAuley. 2018. find
  23. Self-Attentive Sequential Recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pages 197–206. find
  24. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, and 4 other… — Tulu 3: Pushing Frontiers in Open Language Model Post-Training
  25. Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17, page 1419–1428, New York, NY, USA. Association for Computing Machinery. find
  26. Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. 2024. Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page…
  27. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. Towards General Text Embeddings with Multi-stage Contrastive Learning. Preprint, arXiv:2308.03281.
  28. Eugene Ie, Chih-wei Hsu, Martin Mladenov, Vihan Jain, Sanmit Narvekar, Jing Wang, Rui Wu, and Craig Boutilier. 2019. RecSim: A Configurable Simulation Platform for Recommender Systems. Preprint, arXiv:1909.04847. — [arXiv:1909.04847](https://arxiv.org/abs/1909.04847) find
  29. Zenan Li, Fan Nie, Qiao Sun, Fang Da, and Hang Zhao. 2025. Uncertainty-aware decision transformer for stochastic driving environments. In Proceedings of The 8th Conference on Robot Learning, volume 270 of Proceedings of Machine Learning Research, pages 364–386. PMLR. find
  30. Muhammad Intizar Ali, Pankesh Patel, John G. Breslin, Ramy Harik, and Amit Sheth. 2021. Cognitive digital twins for smart manufacturing. IEEE Intelligent Systems, 36(2):96–100. find
  31. Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 2018 world wide web conference, pages 689–698. find
  32. Hongyang Liu, Zhu Sun, Tianjun Wei, Yan Wang, Jiajie Zhu, and Xinghua Qu. 2026. Diagnostic-guided dynamic profile optimization for llm-based user simulators in sequential recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(18):15306–15314. find
  33. Kelong Mao, Jieming Zhu, Jinpeng Wang, Quanyu Dai, Zhenhua Dong, Xi Xiao, and Xiuqiang He. 2021. Simplex: A simple and strong baseline for collaborative filtering. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 1243–1252. find
  34. Muhammad Hasan Maqbool, Umar Farooq, Adib Mosharrof, AB Siddique, and Hassan Foroosh. 2023. Mobilerec: A large scale dataset for mobile apps recommendation. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, pages 3007– 3016. find
  35. Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (…
  36. Mark O’Neill, Elham Vaziripour, Justin Wu, and Daniel Zappala. 2016. Condensing steam: Distilling the diversity of gamer behavior. In Proceedings of the 2016 internet measurement conference, pages 81–95. find
  37. Elizaveta Stavinova, Alexander Grigorievskiy, Anna Volodkevich, Petr Chunaev, Klavdiya Bochenina, and Dmitry Bugaychenko. 2022. Synthetic Data-Based Simulators for Recommender Systems: A Survey. Preprint, arXiv:2206.11338. — [arXiv:2206.11338](https://arxiv.org/abs/2206.11338) find
  38. Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1491…
  39. Zhu Sun, Di Yu, Hui Fang, Jie Yang, Xinghua Qu, Jie Zhang, and Cong Geng. 2020. Are we evaluating rigorously? benchmarking recommendation for reproducible evaluation and fair comparison. In Proceedings of the 14th ACM Conference on Recommender Systems, RecSys ’20, page 23–32, New York, NY, USA. Asso… find
  40. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179… find
  41. Fernando Benjamin Perez Maurera, Maurizio Ferrari Dacrema, Pablo Castells, and Paolo Cremonesi. 2025. Impression-Aware Recommender Systems. ACM Trans. Recomm. Syst. find
  42. Mengting Wan and Julian McAuley. 2018. Item recommendation on monotonic behavior chains. In Proceedings of the 12th ACM Conference on Recommender Systems, RecSys ’18, page 86–94, New York, NY, USA. Association for Computing Machinery. find
  43. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Advances in Neural Information Processing Systems 36, pages 53728–53741. Curran Associates, Inc.
  44. Lei Wang, Jingsen Zhang, Hao Yang, Zhi-Yuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Hao Sun, Ruihua Song, Xin Zhao, Jun Xu, Zhicheng Dou, Jun Wang, and Ji-Rong Wen. 2025. User Behavior Simulation with Large Language Model-based Agents. ACM Trans. Inf. Syst., 43(2):55:1–55:37. find
  45. Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI ’09, pages 452– 461, Arlington, Virginia, USA. AUAI Press. find
  46. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. Preprint, arXiv:2402.03300.
  47. Yifei Shen, Yongji Wu, Yao Zhang, Caihua Shan, Jun Zhang, B Khaled Letaief, and Dongsheng Li. 2021. How powerful is graph convolution for recommendation? In Proceedings of the 30th ACM international conference on information & knowledge management, pages 1619–1629. find
  48. Wenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, and Tat-Seng Chua. 2023a. Diffusion Recommender Model. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’23, pages 832–841, New York, NY, USA. Association for Computing… find
  49. Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pages 165–174. find
  50. Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Yanbin Lu, Xiaojiang Huang, and Yingzhen Yang. 2024a. Rec-Mind: Large Language Model Powered Agent For Recommendation. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 4351–4364, Mexic… find
  51. Yifan Wang, Weizhi Ma, Min Zhang, Yiqun Liu, and Shaoping Ma. 2023b. A Survey on the Fairness of Recommender Systems. ACM Trans. Inf. Syst., 41(3):52:1–52:43. find
  52. Zhenduo Wang, Zhichao Xu, Vivek Srikumar, and Qingyao Ai. 2024b. An In-depth Investigation of User Response Simulation for Conversational Search. In Proceedings of the ACM Web Conference 2024, WWW ’24, pages 1407–1418, New York, NY, USA. Association for Computing Machinery. find
  53. Wei Wei, Quoc Le, Andrew Dai, and Jia Li. 2018. Air-Dialogue: An Environment for Goal-Oriented Dialogue Research. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3844–3854, Brussels, Belgium. Association for Computational Linguistics. find
  54. Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, and Ming Zhou. 2020. MIND: A Large-scale Dataset for News Recommendation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3597–36…
  55. Haotian Wu, Yingpeng Du, Tianjun Wei, Puay Siew Tan, Jie Zhang, Ong Yew Soon, and Zhu Sun. 2026. Efficient large language models for recommendation: A survey. TechRxiv, 2026(0126). find
  56. Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021. Self-supervised Graph Learning for Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, pages 726–735, New York, NY,… find
  57. Xiwang Yang, Harald Steck, Yang Guo, and Yong Liu. 2012. On top-k recommendation using social networks. In Proceedings of the Sixth ACM Conference on Recommender Systems, RecSys ’12, pages 67–74, New York, NY, USA. Association for Computing Machinery. find
  58. An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang, and Tat-Seng Chua. 2024a. On Generative Agents in Recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, pages 1807–1817, New York, NY, USA. Association for Computi… find
  59. Conference on Research and Development in Information Retrieval, SIGIR ’24, pages 2687–2692, New York, NY, USA. Association for Computing Machinery. find
  60. Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2024c. AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems. In Proceedings of the ACM Web Conference 2024, WWW ’24, pages 3679–3689, New York, NY, USA. Ass… find
  61. Zijian Zhang, Shuchang Liu, Ziru Liu, Rui Zhong, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Qidong Liu, and Peng Jiang. 2025. LLM-Powered User Simulator for Recommender System. In Proceedings of the Thirty-Four International Joint Conference on Artificial Intelligence, AAAI ’25. find
  62. Kesen Zhao, Shuchang Liu, Qingpeng Cai, Xiangyu Zhao, Ziru Liu, Dong Zheng, Peng Jiang, and Kun Gai. 2023. KuaiSim: A comprehensive simulator for recommender systems. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, pages 44880–44897, Red Hook,… find
  63. Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, Yingqian Min, Zhichao Feng, Xinyan Fan, Xu Chen, Pengfei Wang, Wendi Ji, Yaliang Li, Xiaoling Wang, and Ji-Rong Wen. 2021a. RecBole: Towards a Unified, Comprehensive and Efficie… find
  64. Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, Yingqian Min, Zhichao Feng, Xinyan Fan, Xu Chen, Pengfei Wang, Wendi Ji, Yaliang Li, Xiaoling Wang, and Ji-Rong Wen. 2021b. Recbole: Towards a unified, comprehensive and efficie… find
  65. Tianwen Zhu, Guangyu Wu, Zhiwei Cao, Ruihang Wang, Jimin Jia, Yong Luo, and Yonggang Wen. 2026. Physics-informed multi-task learning for battery state of health prediction with uncertainty quantification. Proceedings of the AAAI Conference on Artificial Intelligence, 40(2):1659–1667. find
  66. Erhan Zhang, Xingzhu Wang, Peiyuan Gong, Yankai Lin, and Jiaxin Mao. 2024b. USimAgent: Large Language Models for Simulating Search Users. In Proceedings of the 47th International ACM SIGIR A and engagement. It supports research on context-aware recommendations and user experience optimization in mob… find
  67. A.1 Statistics of Source User Feedback Datasets in RS find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wei-2026-mirroring-users-building-preference-aligned/