Elamur Research⌕
☰
home / papers / 2026 / From ID to LLM: Rethinking Representation Learning for Recommendation

From ID to LLM: Rethinking Representation Learning for Recommendation

Paper · 2026 · Song-Li Wu, Zhaocheng Du, Weinan Gan et al.

Summary

The paper examines why embeddings from large language models often lose to classic ID embeddings in recommender systems, even though they carry far more semantic knowledge. Using an information-theoretic argument based on statistical sufficiency and the Blackwell order, the authors prove two things. First, LLM embeddings contain all the predictive information that ID embeddings hold. Second, any deterministic fusion of the two cannot exceed what LLM embeddings already offer.

They attribute the weak empirical results to collaborative signals being tangled up with semantic content and noise. To fix this, they propose Profile-then-Embedding (PtE), which has two stages:

  • Profile Stage: a fine-tuned LLaMa3-8B writes natural-language profiles of users and items. It reasons iteratively in both directions over interaction histories and the neighbours of each user.
  • Personalized Embedding Stage: these profiles are turned into compact embeddings optimised for the recommendation task, including a GRPO-based alignment step.

On three benchmarks, including cold-start and long-tail settings, PtE improves both discriminative and generative recommenders and beats fusion approaches. This suggests LLM representations could serve as a standalone backbone for recommendation.

Key points

  • Theory and case study: an Information Sufficiency Index (ISI) supports the theory. In a case study comparing SASRec ID embeddings, LLaMa3-8B embeddings and an MLP-based fusion, LLM embeddings capture over 95% of the fusion's information. The fusion recovers under 85% of the LLM's information.
  • Neighbourhoods for user profiles: collaborative neighbours are chosen by an interest-overlap ratio, defined as the share of a user's items that another user also interacted with.
  • Cold-start ablation (SASRec backbone): on Amazon Movies, the full PtE reaches N@10 = 0.0523. Embedding LLM outputs directly without profiles gives 0.0451, the largest drop of any variant.
  • Long-tail ablation: on Yelp, tail-item R@10 is 0.2009 for the full model versus 0.1523 without profiles. Removing collaborative alignment from GRPO hurts tail items the most.
  • Profiling and limitations: user-only, item-only and single-pass profiling all fall short of the full model. The authors acknowledge that the iterative profile generation adds computational cost.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓University of Science and Technology of ChinaHefei · China145 works · 114,409 citations🏢Huawei TechnologiesShenzhen · China96 works · 15,432 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksInformation Systems · 1,730 worksDecision Sciences · 249 worksManagement Science and Operations Research · 134 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Song-Li Wu, Zhaocheng Du, Weinan Gan, Jingyi Wang and Xianquan Wang. 2026. From ID to LLM: Rethinking Representation Learning for Recommendation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.179

@inproceedings{wu2026id,
  title = {{From ID to LLM: Rethinking Representation Learning for Recommendation}},
  author = {Song-Li Wu, Zhaocheng Du, Weinan Gan, Jingyi Wang and Xianquan Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.179},
  url = {https://doi.org/10.18653/v1/2026.acl-long.179}
}

References

42 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Guillaume Alain and Yoshua Bengio. 2016. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644. find
  2. David Blackwell. 1953. Equivalent comparisons of experiments. The annals of mathematical statistics, pages 265–272. find
  3. George Casella and Roger Berger. 2024. Statistical inference. Chapman and Hall/CRC. find
  4. Gaode Chen, Ruina Sun, Yuezihan Jiang, Jiangxia Cao, Qi Zhang, Jingjian Lin, Han Li, Kun Gai, and Xinghua Zhang. 2024. A multi-modal modeling framework for cold-start short-video recommendation. In Proceedings of the 18th ACM Conference on Recommender Systems, pages 391–400. find
  5. Lei Chen, Chen Gao, Xiaoyi Du, Hengliang Luo, Depeng Jin, Yong Li, and Meng Wang. 2025. Enhancing id-based recommendation with large language models. ACM Transactions on Information Systems, 43(5):1–30. find
  6. Jesse Harte, Wouter Zorgdrager, Panos Louridas, As-terios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. 2023. Leveraging large language models for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems, pages 1096–1102. find
  7. Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 639–648. find
  8. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840– 6851. find
  9. Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. 2024. Bridging language and items for retrieval and recommendation. arXiv preprint arXiv:2403.03952. — [arXiv:2403.03952](https://arxiv.org/abs/2403.03952) find
  10. Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 585–593. find
  11. Guoqing Hu, Zhengyi Yang, Zhibo Cai, An Zhang, and Xiang Wang. 2024a. Generate and instantiate what you prefer: Text-guided diffusion for sequential recommendation. arXiv preprint arXiv:2410.13428. — [arXiv:2410.13428](https://arxiv.org/abs/2410.13428) find
  12. Guoqing Hu, An Zhang, Shuo Liu, Zhibo Cai, Xun Yang, and Xiang Wang. 2025. Alphafuse: Learn id embeddings for sequential recommendation in null space of language embeddings. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1614… find
  13. Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. 2024b. Enhancing sequential recommendation via llm-based semantic embedding learning. In Companion Proceedings of the ACM Web Conference 2024, pages 103–111. find
  14. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pages 197–206. IEEE. find
  15. Kenneth Li, Aspen K Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2022. Emergent world representations: Exploring a sequence model trained on a synthetic task. arXiv preprint arXiv:2210.13382. find
  16. Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Hao Zhang, Yong Liu, Chuhan Wu, Xiangyang Li, Chenxu Zhu, and 1 others. 2025. How can recommender systems benefit from large language models: A survey. ACM Transactions on Information Systems, 43(2):1–47.
  17. Qidong Liu, Xian Wu, Wanyu Wang, Yejing Wang, Yuanshao Zhu, Xiangyu Zhao, Feng Tian, and Yefeng Zheng. 2025. Llmemb: Large language model can be a good embedding generator for sequential recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 12183– 12191. find
  18. Qidong Liu, Xian Wu, Xiangyu Zhao, Yejing Wang, Zijian Zhang, Feng Tian, and Yefeng Zheng. 2024. Large language models enhanced sequential recommendation for long-tail user and item. arXiv e-prints, pages arXiv–2405. find
  19. Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 43–52. find
  20. Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. 2022. Linearly mapping from image to text space. arXiv preprint arXiv:2209.15162. find
  21. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
  22. Roma Patel and Ellie Pavlick. 2022. Mapping language models to grounded conceptual spaces. In International conference on learning representations. find
  23. Zekai Qu, Ruobing Xie, Chaojun Xiao, Zhanhui Kang, and Xingwu Sun. 2024. The elephant in the room: rethinking the usage of pre-trained language model in sequential recommendation. In Proceedings of the 18th ACM Conference on Recommender Systems, pages 53–62. before recommend: Unleashing the latent r… — [arXiv:2503.22675](https://arxiv.org/abs/2503.22675) find
  24. Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, and Anna Korhonen. 2020. Probing pretrained language models for lexical semantics. arXiv preprint arXiv:2010.05731.
  25. Yaqing Wang, Hongming Piao, Daxiang Dong, Quanming Yao, and Jingbo Zhou. 2024. Warming up cold-start ctr prediction by learning item-specific feature interactions. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3233–3244. find
  26. Yu Wang, Lei Sang, Yi Zhang, and Yiwen Zhang. 2025a. Intent representation learning with large language model for recommendation. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1870–1879. find
  27. Xubin Ren, Wei Wei, Lianghao Xia, Lixin Su, Suqi Cheng, Junfeng Wang, Dawei Yin, and Chao Huang. 2024. Representation learning with large language models for recommendation. In Proceedings of the ACM web conference 2024, pages 3464–3475. find
  28. Yuhao Wang, Junwei Pan, Xinhang Li, Maolin Wang, Yuan Wang, Yue Liu, Dapeng Liu, Jie Jiang, and Xiangyu Zhao. 2025b. Empowering large language model for sequential recommendation via multimodal embeddings and semantic ids. In Proceedings of the 34th ACM International Conference on Information and Kn… find
  29. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.
  30. Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021. Self-supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 726–735. find
  31. Leheng Sheng, An Zhang, Yi Zhang, Yuxin Chen, Xiang Wang, and Tat-Seng Chua. 2024. Language models encode collaborative signals in recommendation. find
  32. Song-Li Wu, Liang Du, Jia-Qi Yang, Yu-Ai Wang, De-Chuan Zhan, Shuang Zhao, and Zi-Xun Sun. 2023. Re-sort: Removing spurious correlation in multi-level interaction for ctr prediction. arXiv preprint arXiv:2309.14891. — [arXiv:2309.14891](https://arxiv.org/abs/2309.14891) find
  33. Hongyi Shui, Yuanjing Zhu, Fan Zhuo, Yibo Sun, and Daoyuan Li. 2024. An emotion text classification model based on llama3-8b using lora technique. In 2024 7th International Conference on Computer Information Science and Application Technology (CISAT), pages 380–383. IEEE. find
  34. Zhengyi Yang, Jiancan Wu, Zhicai Wang, Xiang Wang, Yancheng Yuan, and Xiangnan He. 2023. Generate what you prefer: Reshaping sequential recommendation via guided diffusion. Advances in Neural Information Processing Systems, 36:24247–24261. find
  35. Anima Singh, Trung Vu, Nikhil Mehta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Devansh Tandon, and 1 others. 2024. Better generalization with semantic ids: A case study in ranking for recommendations. In Proceedings of the 18th ACM Conference on… find
  36. Zheng Yuan, Fajie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang, Yunzhu Pan, and Yongxin Ni. 2023. Where to go next for recommender systems? id-vs. modality-based recommender models revisited. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information R… find
  37. Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021. Whitening sentence representations for better semantics and faster retrieval. arXiv preprint arXiv:2103.15316. find
  38. Jirong Zha, Yuxuan Fan, Xiao Yang, Chen Gao, and Xinlei Chen. 2025. How to enable llm with 3d capacity? a survey of spatial reasoning in llm. arXiv preprint arXiv:2504.05786. — [arXiv:2504.05786](https://arxiv.org/abs/2504.05786) find
  39. Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Jian Wu, and Yuning Jiang. 2025. find
  40. Think Junjie Zhang, Ruobing Xie, Yupeng Hou, Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2025. Recommendation as instruction following: A large language model empowered recommendation approach. ACM Transactions on Information Systems, 43(5):1–37. find
  41. Lingzi Zhang, Xin Zhou, Zhiwei Zeng, and Zhiqi Shen. 2024. Are id embeddings necessary? whitening pre-trained text embeddings for effective sequential recommendation. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 530– 543. IEEE. find
  42. Yongchun Zhu, Ruobing Xie, Fuzhen Zhuang, Kaikai Ge, Ying Sun, Xu Zhang, Leyu Lin, and Juan Cao. 2021. Learning to warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In Proceedings of the 44th International ACM SIGIR Conference on Research and Develop… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2026-id-llm-rethinking-representation-learning/