Elamur Research⌕
☰
home / papers / 2026 / From Past To Path: Masked History Learning for Next-Item Prediction in Generative Recommendation

From Past To Path: Masked History Learning for Next-Item Prediction in Generative Recommendation

Paper · 2026 · Kaiwen Wei, Kejun he, Xiaomian Kang et al.

Summary

Generative recommenders produce semantic item IDs one step at a time. This left-to-right training tends to anchor them on the most recent interactions and miss longer-range user intent. To counter this, the authors introduce Masked History Learning (MHL), which changes how decoder-only generative models are trained. Alongside standard next-item prediction, the model must also recover items hidden in the user's past sequence, which pushes it to model why a sequence of purchases forms, not only which item follows.

Two additions refine the approach:

  • an entropy-guided policy that masks the most informative history positions instead of random ones;
  • a curriculum that starts with random-mask warmup, moves to heavy entropy-based masking, and then reduces masking so training matches autoregressive inference.

Experiments on three Amazon Reviews 2014 categories show MHL beating state-of-the-art generative baselines on Recall@K and NDCG@K. It also stays robust when recent history is truncated, handles cold-start items better, and transfers to text-token sequences. The work suggests that richer training objectives, rather than new architectures, can make generative recommenders less short-sighted.

Key points

  • Evaluation uses the Beauty, Toys and Sports categories of Amazon Reviews 2014, with RPG as the main generative baseline. Code is released at github.com/CQU-MM-Intelligent-Lab/MHL.
  • In a history-truncation test, MHL stays robust and beats the leading baseline by more than 40%, while existing SOTA models degrade sharply.
  • Masking works at item, token and mixed granularity, and all three give gains. Mixed masking combines harder item-level and easier token-level reconstruction.
  • The reconstruction loss weight works best in the 1.0–1.8 range (prediction loss fixed at 1.0). Performance drops below 0.8 and also when the weight becomes too large.
  • In cold-start bins ([0,5] to [16,20] training occurrences), MHL outperforms RPG in every bin, with the largest margin on the rarest items. Applied to raw title tokens on Beauty, MHL also outperforms RPG.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Chinese Academy of SciencesBeijing · China194 works · 23,906 citations🔬Institute of AutomationBeijing · China70 works · 14,251 citations🎓City University of Hong KongHong Kong · China45 works · 3,014 citations🎓Chongqing UniversityChongqing · China7 works · 318 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

Cite

Cite as Kaiwen Wei, Kejun he, Xiaomian Kang, Jie Zhang, Ymyang, Li Jin, Zhenyang Li, Jiang Zhong, Richard He Bai and Junnan Zhu. 2026. From Past To Path: Masked History Learning for Next-Item Prediction in Generative Recommendation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.475

@inproceedings{wei2026past,
  title = {{From Past To Path: Masked History Learning for Next-Item Prediction in Generative Recommendation}},
  author = {Kaiwen Wei, Kejun he, Xiaomian Kang, Jie Zhang, Ymyang, Li Jin, Zhenyang Li, Jiang Zhong, Richard He Bai and Junnan Zhu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.475},
  url = {https://doi.org/10.18653/v1/2026.acl-long.475}
}

References

44 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys), pages 1007–1014. find
  2. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning (ICML), pages 41–48.
  3. Jianxin Chang, Chen Gao, Yu Zheng, Yiqun Hui, Yanan Niu, Yang Song, Depeng Jin, and Yong Li. 2021. Sequential recommendation with graph neural networks. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 378–387. find
  4. Rui Chen, Qingyi Hua, Yan-Shuo Chang, Bo Wang, Lei Zhang, and Xiangjie Kong. 2018. A survey of collaborative filtering-based recommender systems: From traditional methods to hybrid methods based on social networks. IEEE Access, 6:64301–64320. find
  5. Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524. — [arXiv:2303.14524](https://arxiv.org/abs/2303.14524) find
  6. Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. 2014. Optimized product quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 36(4):744–755. find
  7. Yongjing Hao, Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor S. Sheng, Jiajie Xu, Guanfeng Liu, and Xiaofang Zhou. 2023. Feature-level deeper self-attention network with contrastive learning for sequential recommendation. IEEE Transactions on Knowledge and Data Engineering (TKDE), 35(10):10112–10… find
  8. Jesse Harte, Wouter Zorgdrager, Panos Louridas, As-terios Katsifodimos, Dietmar Jannach, and Marios Fragkoulis. 2023. Leveraging large language models for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys), pages 1096–1102. find
  9. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 770–778.
  10. Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web (WWW), pages 507–517. find
  11. Zhicheng He, Weiwen Liu, Wei Guo, Jiarui Qin, Yingxue Zhang, Yaochen Hu, and Ruiming Tang. 2023. A survey on user behavior modeling in recommender systems. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI, pages 6656–6664. find
  12. Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In 4th International Conference on Learning Representations (ICLR). find
  13. Yupeng Hou, Zhankui He, Julian McAuley, and Wayne Xin Zhao. 2023. Learning vector-quantized item representation for transferable sequential recommenders. In Proceedings of the ACM Web Conference 2023 (WWW), pages 1162–1171. find
  14. Yupeng Hou, Jiacheng Li, Ashley Shin, Jinsung Jeon, Abhishek Santhanam, Wei Shao, Kaveh Hassani, Ning Yao, and Julian McAuley. 2025a. Generating long semantic ids in parallel for recommendation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 956–… find
  15. Yupeng Hou, An Zhang, Leheng Sheng, Zhengyi Yang, Xiang Wang, Tat-Seng Chua, and Julian McAuley. 2025b. Generative recommendation models: Progress and directions. In Companion Proceedings of the ACM on Web Conference 2025 (WWW), pages 13–16. find
  16. Wenyue Hua, Shuyuan Xu, Yingqiang Ge, and Yongfeng Zhang. 2023. How to index item ids for recommendation foundation models. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP), pages 195–204. find
  17. Sara: Salience-aware reinforced adaptive decoding for large language models in abstractive summarization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 25450–25463. find
  18. Chen Ma, Peng Kang, and Xue Liu. 2019. Hierarchical gating networks for sequential recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD), pages 825–833. find
  19. David J. C. MacKay. 2002. Information Theory, Inference & Learning Algorithms. Cambridge University Press, USA. find
  20. Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 43–52. find
  21. Niklas Muennighoff, Hongjin SU, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2025. Generative representational instruction tuning. In The Thirteenth International Conference on Learning Representations (ICLR).
  22. Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864–1874.
  23. Aleksandr V. Petrov and Craig Macdonald. 2024. Recjpq: Training large-catalogue sequential recommenders. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining (WSDM), pages 538–547. find
  24. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pages 197–206. find
  25. Erasmo Purificato, Ludovico Boratto, and Ernesto William De Luca. 2024. User modeling and user profiling: A comprehensive survey. arXiv preprint arXiv:2402.09660. — [arXiv:2402.09660](https://arxiv.org/abs/2402.09660) find
  26. Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM), pages 1419–1428. find
  27. Shashank Rajput, Nikhil Mehta, Anima Singh, Raghu-nandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathiamoorthy. 2023. Recommender systems with generative retrieval. In Thirty-seventh Conference on Neural Inform… find
  28. Pang Li, Shahrul Azman Mohd Noah, and Hafiz Mohd Sarim. 2024. A survey on deep neural networks in collaborative filtering recommendation systems. arXiv preprint arXiv:2412.01378. — [arXiv:2412.01378](https://arxiv.org/abs/2412.01378) find
  29. Haitao Lin, Zhen Yang, Jiawei Xue, Ziji Zhang, Luzhu Wang, Yikun Gu, Yao Xu, and Xin Li. 2025. Spacetime-gr: A spacetime-aware generative model for large scale online poi recommendation. arXiv preprint arXiv:2508.16126. — [arXiv:2508.16126](https://arxiv.org/abs/2508.16126) find
  30. Nayu Liu, Junnan Zhu, Yiming Ma, Zhicong Lu, Wenlei Xu, Yong Yang, Jiang Zhong, and Kaiwen Wei. 2025. find
  31. Jun Rao, Xuebo Liu, Hexuan Deng, Zepeng Lin, Zix-iong Yu, Jiansheng Wei, Xiaojun Meng, and Min Zhang. 2026. Dynamic sampling that adapts: Self-aware iterative data persistent optimization for mathematical reasoning. In Findings of the Association for Computational Linguistics: ACL 2026.
  32. Jun Rao, Xuebo Liu, Lian Lian, Shengjun Cheng, Yunjie Liao, and Min Zhang. 2024. CommonIT: Commonality-aware instruction tuning for large language models via data partitions. In EMNLP, pages 10064–10083, Miami, Florida, USA. Association for Computational Linguistics. find
  33. Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW), pages 811–820. find
  34. Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM), pages 1441–1450. find
  35. Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (WSDM), pages 565–573. find
  36. Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. 2022. Transformer memory as a differentiable search index. In Proceedings of the 36th International Conference on Neural Information…
  37. Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015. Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 1235–1244. find
  38. Wenjie Wang, Honghui Bao, Xinyu Lin, Jizhi Zhang, Yongqi Li, Fuli Feng, See-Kiong Ng, and Tat-Seng Chua. 2024. Learnable item tokenization for generative recommendation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (CIKM), pages 2400–2409. find
  39. Yujing Wang, Yingyan Hou, Haonan Wang, Ziming Miao, Shibin Wu, Hao Sun, Qi Chen, Yuqing Xia, Chengmin Chi, Guoshuai Zhao, Zheng Liu, Xing Xie, Hao Allen Sun, Weiwei Deng, Qi Zhang, and Mao Yang. 2022. A neural corpus indexer for document retrieval. In Proceedings of the 36th International Conference… find
  40. Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based recommendation with graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 346–353. find
  41. Zheng Yuan, Fajie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang, Yunzhu Pan, and Yongxin Ni. 2023. Where to go next for recommender systems? id- vs. modality-based recommender models revisited. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information… find
  42. Zhenrui Yue, Yueqi Wang, Zhankui He, Huimin Zeng, Julian Mcauley, and Dong Wang. 2024. Linear recurrent units for sequential recommendation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining (WSDM), pages 930–938. find
  43. Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. 2024. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. In Proceedings of the 41st International Conference… find
  44. Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Man… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wei-2026-past-path-masked-history-learning/