Elamur Research⌕
☰
home / papers / 2026 / GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model Recommendation

GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model Recommendation

Paper · 2026 · Lin Mu, Guoji Wang, Li Ni et al.

Summary

LLM-based recommenders have trouble combining a language model's textual knowledge with collaborative signals from user–item interactions, which are naturally graph-shaped. Earlier approaches either turn interaction histories into prompt text or soft tokens (TALLRec, CoLLM), or insert fixed, externally trained embeddings into adapter weights (CoRA). In both cases the structural information stays static, and multi-hop relations are lost. GraphLoRA instead places a trainable message-passing GNN inside the LoRA bottleneck, between matrices A and B. Collaborative embeddings, initialized from matrix factorization, are projected into the rank-r space. There they are aggregated over neighbours in the user–item bipartite graph, then mapped back into the model through B. The graph encoder and the LoRA adapter are trained together under the recommendation objective. Experiments on ML-1M, Amazon-Book and Book-Crossing, using Vicuna and LLaMA-7B backbones, show that it outperforms state-of-the-art LLM recommenders with only marginal extra cost. The work suggests that graph reasoning can be built into the adaptation parameters themselves, which gives a cheap path to structure-aware fine-tuning.

Key points

  • Core idea: the method extends LoRA from independent low-rank updates to "structure-aware low-rank propagation." The graph module operates in the small rank-r space rather than at d_model width.
  • Efficiency: relative to a LoRA-only baseline with trainable MF, GraphLoRA uses 1.017× the parameters and adds roughly 1% latency. CoRA-MF needs 7.635× the parameters, and injecting the signal before A (Pre-A) costs 5.224×.
  • Ablations: freezing the MF embeddings gives the weakest results, making them trainable helps, and adding graph aggregation helps most. Among GCN, LightGCN and NGCF encoders, NGCF performs best.
  • Injection depth: placing the GNN in deeper Transformer layers works better. The best results come from layer 31 on ML-1M and layer 15 on Amazon-Book.
  • Few-shot results: on Book-Crossing with LLaMA-7B under the TALLRec protocol, GraphLoRA reaches AUC 60.90, 61.08 and 67.90 with 16, 64 and 256 training samples. TALLRec scores 56.36, 60.39 and 64.38 on the same settings. Code is available at github.com/wgj15965/GraphLoRA.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Science and Technology of ChinaHefei · China108 works · 112,698 citations🎓Anhui UniversityHefei · China7 works · 7 citations🎓Hefei UniversityHefei · China3 works · 1 citation

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEfficient models & compression · 242 works

Cite

Cite as Lin Mu, Guoji Wang, Li Ni, Lei Sang, Zhize Wu, Peiquan Jin and Yiwen Zhang. 2026. GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model Recommendation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.645

@inproceedings{mu2026graphlora,
  title = {{GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model Recommendation}},
  author = {Lin Mu, Guoji Wang, Li Ni, Lei Sang, Zhize Wu, Peiquan Jin and Yiwen Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.645},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.645}
}

References

39 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. TALLRec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys), RecSys ’23, pages 1007–1014. Associati… find
  2. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing GPT-4 with 90%* Chat-GPT quality. find
  3. Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongxiang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering chatgpt’s capabilities in recommender systems. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys), pages 1126–1132. find
  4. Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220–235.
  5. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. A survey on in-context learning. Preprint, arXiv:2301.00234.
  6. Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv preprint arXiv:2303.14524. — [arXiv:2303.14524](https://arxiv.org/abs/2303.14524) find
  7. Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In Proceedings of the 16th ACM Conference on Recommender Systems (RecSys), pages 299–315. find
  8. Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning (ICML).
  9. Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems (NeurIPS), volume 30.
  10. F. Maxwell Harper and Joseph A. Konstan. 2015. The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems, 5(4):1–19. find
  11. Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web, pages 507–517. find
  12. Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 639–648. find
  13. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning (ICML), pages 2790–2799. PMLR. find
  14. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR).
  15. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2024. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems.
  16. Mingyu Jin, Qinkai Yu, Jingyuan Huang, Qingcheng Zeng, Zhenting Wang, Wenyue Hua, Haiyan Zhao, Kai Mei, Yanda Meng, Kaize Ding, Fan Yang, Mengnan Du, and Yongfeng Zhang. 2025. Exploring concept depth: How large language models acquire knowledge and concept at different layers? In Proceedings of the…
  17. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pages 197–206. IEEE. find
  18. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR).
  19. Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37. find
  20. Lei Li, Yongfeng Zhang, and Li Chen. 2023. Personalized prompt learning for explainable recommendation. ACM Transactions on Information Systems, 41(4):1–26. find
  21. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), pages 4582–4597. find
  22. Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. 2024. LLaRA: Large language-recommendation assistant. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 1785– 1795. find
  23. Qidong Liu, Xian Wu, Wanyu Wang, Yejing Wang, Yuanshao Zhu, Xiangyu Zhao, Feng Tian, and Yefeng Zheng. 2025a. LLMEmb: Large language model can be a good embedding generator for sequential recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 39, pages 12183–… find
  24. Yiyu Liu, Qian Liu, Yu Tian, Changping Wang, Yanan Niu, Yang Song, and Chenliang Li. 2021. Concept-aware denoising graph neural network for micro-video recommendation. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management, pages 1099–1108. find
  25. Yuting Liu, Jinghao Zhang, Yizhou Dang, Yuliang Liang, Qiang Liu, Guibing Guo, Jianzhe Zhao, and Xingwei Wang. 2025b. Cora: Collaborative information perception by large language model’s weights for recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 39, p… find
  26. Lin Mu, Xiaoyu Wang, Li Ni, Yang Li, Zhize Wu, Peiquan Jin, and Yiwen Zhang. 2025. DenseLoRA: Dense low-rank adaptation of large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 10198–10211. Association for Computational Linguistics. find
  27. Oscar Skean, Md Rifat Arefin, Yann LeCun, and Ravid Shwartz-Ziv. 2024. Does representation matter? exploring intermediate layers in large language models. arXiv preprint arXiv:2412.09563. — [arXiv:2412.09563](https://arxiv.org/abs/2412.09563) find
  28. Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2024a. GraphGPT: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 491–500. find
  29. ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 165– 174. find
  30. Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al. 2024. A survey on large language models for recommendation. World Wide Web, 27(5):60. find
  31. Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. 2022. Graph neural networks in recommender systems: A survey. ACM Computing Surveys, 55(5):1–37. find
  32. Yang Zhang, Keqin Bao, Ming Yan, Wenjie Wang, Fuli Feng, and Xiangnan He. 2024. Text-like encoding of collaborative information in large language models for recommendation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 9061–9079. find
  33. Yang Zhang, Fuli Feng, Jizhi Zhang, Keqin Bao, Qifan Wang, and Xiangnan He. 2025. Collm: Integrating collaborative embeddings into large language models for recommendation. IEEE Transactions on Knowledge and Data Engineering, 37(5):2329–2340. find
  34. Zizhuo Zhang and Bang Wang. 2023. Prompt learning for news recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 1157–1166. find
  35. Haiteng Zhao, Shengchao Liu, Chang Ma, Hannan Xu, Jie Fu, Zhi-Hong Deng, Lingpeng Kong, and Qi Liu. 2023. Gimlet: A unified graph-text model for instruction-based molecule zero-shot learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 36, pages 6485–6509. find
  36. Cai-Nicolas Ziegler. 2005. Improving recommendation lists through topic diversification. In WWW’05: Proceedings of the 14th international conference on World Wide Web, pages 22–32. ACM. find
  37. Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. 2024b. HiGPT: Heterogeneous graph language model. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pages 2842– 2853. find
  38. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In International Conference on Learning Representations (ICLR). find
  39. Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the 42nd International find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mu-2026-graphlora-structure-aware-low-rank/