Elamur Research⌕
☰
home / papers / 2026 / LeLoRA: Learnable Low-Rank Adaptation of Large Language Models

LeLoRA: Learnable Low-Rank Adaptation of Large Language Models

Paper · 2026 · Xiaoling Zhou, Mingjie Zhang, Zhemg Lee et al.

Summary

LoRA-style fine-tuning usually relies on hand-set, fixed choices about which parts of each weight matrix to train, such as a uniform rank or always the principal or minor singular directions. The authors argue this wastes parameters and limits accuracy. Their framework, LeLoRA, factorizes each pre-trained weight matrix with SVD once before training. A small policy network then decides, separately for every matrix, two things: how many singular components to train (the effective dimension r) and where in the spectrum that block begins (the location index s). The policy's inputs are matrix descriptors such as module type, layer, singular values and norm. The LLM and the policy are trained in alternation, with the policy updated by REINFORCE using rewards derived from the LLM's loss, so the selection keeps adjusting as training progresses. The method was tested on ten LLMs (125M–70B parameters) across NLU, commonsense reasoning, math, code generation and instruction following. It beat existing baselines while often training far fewer parameters. This makes the work relevant to anyone fine-tuning models on a limited compute budget.

Key points

  • The policy network outputs a per-matrix pair (r, s). For example, r = 8 and s = 2048 selects singular components 2048–2055 for training and freezes the rest.

  • On commonsense reasoning with LLaMA2-7B, LeLoRA scored 81.0 with 25M tunable parameters, versus LoRA at 77.6 (56M) and DoRA at 79.7 (57M). In a larger setting it reached 81.6 with 39M parameters, while DoRA scored 79.4 with 113M.

  • In experiments on MetaMathQA-100K, evaluated on GSM8K, learned positions clearly beat fixed positions (primary, medium or minor components). Larger candidate sets for the dimension also improved results.

  • The learned strategies follow clear patterns:

    • The trained dimension grows over the course of training.
    • Deeper layers receive more trainable components than shallower ones.
    • Feed-forward Up/Down matrices dominate early training, while attention matrices take over later.

  • The authors report three limitations:

    • The REINFORCE-based policy training adds computational overhead.
    • Only four matrix descriptors are used as policy inputs.
    • Only two strategy variables are learned (dimension and location).

    Results were stable for the loss-weighting factor α in [0.5, 1].

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations🎓Tianjin UniversityTianjin · China41 works · 1,128 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEfficient models & compression · 242 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Latent semantic indexing · 29 works

Cite

Cite as Xiaoling Zhou, Mingjie Zhang, Zhemg Lee, Wei Ye and Shikun Zhang. 2026. LeLoRA: Learnable Low-Rank Adaptation of Large Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1628

@inproceedings{zhou2026lelora,
  title = {{LeLoRA: Learnable Low-Rank Adaptation of Large Language Models}},
  author = {Xiaoling Zhou, Mingjie Zhang, Zhemg Lee, Wei Ye and Shikun Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1628},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1628}
}

References

99 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Paul Albert, Frederic Z. Zhang, Hemanth Saratchandran, Cristian Rodriguez-Opazo, Anton van den Hengel, and Ehsan Abbasnejad. 2025. Randlora: Full rank parameter-efficient fine-tuning of large models. In The Thirteenth International Conference on Learning Representations. find
  3. Sercan Ö Arik and Tomas Pfister. 2021. Tabnet: Attentive interpretable tabular learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6679–6687. find
  4. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
  5. Klaudia Bałazy, Mohammadreza Banaei, Karl Aberer, and Jacek Tabor. 2024. Lora-xs: Low-rank adaptation with extremely small number of parameters. arXiv preprint arXiv:2405.17604. — [arXiv:2405.17604](https://arxiv.org/abs/2405.17604) find
  6. Luisa Bentivogli, Ido Dagan, and Bernardo Magnini. 2017. The recognizing textual entailment challenges: Datasets and methodologies. Handbook of Linguistic Annotation, pages 1119–1147. find
  7. Gaurav Bhatnagar, QM Jonathan Wu, and Pradeep K Atrey. 2013. Secure randomized image watermarking based on singular value decomposition. ACM Transactions on Multimedia Computing, Communications, and Applications, 10(1):1–21. find
  8. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7432–7439.
  9. Albrecht Böttcher and David Wenzel. 2008. The frobenius norm and the commutator. Linear Algebra and Its Applications, 429(8-9):1864–1885. find
  10. Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada.…
  11. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  12. Yuyan Chen, Qiang Fu, Ge Fan, Lun Du, Jian-Guang Lou, Shi Han, Dongmei Zhang, Zhixu Li, and Yanghua Xiao. 2023. Hadamard adapter: An extreme parameter-efficient adapter tuning method for pre-trained language models. In Proceedings of the 32nd ACM International Conference on Information and Knowledge… find
  13. Zhiwei Chen, Yupeng Hu, Zixu Li, Zhiheng Fu, Xuemeng Song, and Liqiang Nie. 2025. Offset: Segmentation-based focus shift revision for composed image retrieval. In Proceedings of the ACM International Conference on Multimedia, page 6113–6122. find
  14. Kexin Chu, Zixu Shen, Dawei Xiang, and Wei Zhang. 2025. Safekv: Safe kv-cache sharing in llm serving. In Machine Learning for Computer Architecture and Systems. find
  15. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:…
  16. Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  17. Carl Eckart and Gale Young. 1936. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211–218. find
  18. Soufiane Hayou, Nikhil Ghosh, and Bin Yu. 2024. Lora+: Efficient low rank adaptation of large models. In Proceedings of the 41st International Conference on Machine Learning, pages 17783–17806. find
  19. Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2022. Towards a unified view of parameter-efficient transfer learning. In Proceedings of the 10th International Conference on Learning Representations.
  20. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision, pages 1026–1034. find
  21. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770– 778.
  22. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks.
  23. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457.
  24. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In Proceedings of 10th International Conference on Learning Representations.
  25. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  26. Yuxuan Hu, Jing Zhang, Xiaodong Chen, Zhe Zhao, Cuiping Li, and Hong Chen. 2025. Lors: Efficient low-rank adaptation for sparse large language model. arXiv preprint arXiv:2501.08582. — [arXiv:2501.08582](https://arxiv.org/abs/2501.08582) find
  27. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. Ultrafeedback: Boosting language models with scaled AI feedback. In Proceedings of the 41st International Conference on Machine Learning, pages 97… find
  28. Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee. 2023. LLM-adapters: An adapter family for parameter-efficient fine-tuning of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,…
  29. William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005).
  30. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and 1 others. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. find
  31. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hann… find
  32. Jiantong Jiang, Peiyu Yang, Rui Zhang, and Feng Liu. 2025a. Towards efficient large language model serving: A survey on system-aware kv cache optimization. Authorea Preprints. find
  33. Ting Jiang, Shaohan Huang, Shengyue Luo, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, Deqing Wang, and 1 others. 2024b. Mora: High-rank updating for parameter-efficient fine-tuning. arXiv preprint arXiv:2405.12130. — [arXiv:2405.12130](https://arxiv.org/abs/2405.12130) find
  34. Yuxuan Jiang and Francis Ferraro. 2026a. Beyond math: Stories as a testbed for memorization-constrained reasoning in llms. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5590–5607.
  35. Yuxuan Jiang and Francis Ferraro. 2026b. Scribe: Structured mid-level supervision for tool-uing language models. arXiv preprint arXiv:2601.03555. — [arXiv:2601.03555](https://arxiv.org/abs/2601.03555) find
  36. Yuxuan Jiang, Dawei Li, and Frank Ferraro. 2025b. Drp: Distilled reasoning pruning with skill-aware step decomposition for efficient large reasoning models. arXiv preprint arXiv:2505.13975.
  37. Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, and James Henderson. 2021. Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conf…
  38. Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki M Asano. 2024. Vera: Vector-based random matrix adaptation. In The Twelfth International Conference on Learning Representations.
  39. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. find
  40. Zixu Li, Yupeng Hu, Zhiwei Chen, Qinlei Huang, Guozhi Qiu, Zhiheng Fu, and Meng Liu. 2026a. Retrack: Evidence-driven dual-stream directional anchor calibration network for composed video retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 23373–23381. find
  41. Zixu Li, Yupeng Hu, Zhiwei Chen, Shiqi Zhang, Qinlei Huang, Zhiheng Fu, and Yinwei Wei. 2026b. Habit: Chrono-synergia robust progressive learning framework for composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6762–6770. find
  42. Debin Liu, Laurence T Yang, Puming Wang, Ruonan Zhao, and Qingchen Zhang. 2022a. Tt-tsvd: A multimodal tensor train decomposition with its application in convolutional neural networks for smart healthcare. ACM Transactions on Multimedia Computing, Communications, and Applications, 18(1s):1–17. find
  43. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. Dora: Weight-decomposed low-rank adaptation. In Proceedings of the 41st International Conference on Machine Learning, pages 32100–32121. find
  44. Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022b. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61–…
  45. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  46. Xueqi Ma, Shuo Yang, Yanbei Jiang, Shu Liu, Zhenzhen Liu, Jiayang Ao, Xingjun Ma, Sarah Monazam Erfani, and James Bailey. 2026. Attention in space: Functional roles of vlm heads for spatial reasoning. arXiv preprint arXiv:2603.20662. — [arXiv:2603.20662](https://arxiv.org/abs/2603.20662) find
  47. Reem I Masoud, Martin Ferianc, Philip Colin Treleaven, and Miguel RD Rodrigues. 2024. Llm alignment using soft prompt tuning: The case of cultural alignment. In Workshop on Socially Responsible Language Modelling Research. find
  48. Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. Pissa: Principal singular values and singular vectors adaptation of large language models. In The 38th Annual Conference on Neural Information Processing Systems. find
  49. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391, Brussels, Belgium. Association fo… find
  50. Lin Mu, Xiaoyu Wang, Li Ni, Yang Li, Zhize Wu, Pei-quan Jin, and Yiwen Zhang. 2025. Denselora: Dense low-rank adaptation of large language models. arXiv preprint arXiv:2505.23808. find
  51. Rui Pan, Xiang Liu, Shizhe Diao, Renjie Pi, Jipeng Zhang, Chi Han, and Tong Zhang. 2024. Lisa: Layer-wise importance sampling for memory-efficient large language model fine-tuning. In Proceedings of the 38th International Conference on Neural Information Processing Systems, pages 57018–57049. find
  52. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789. Association for Computational Linguistics.
  53. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  54. Pengjie Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, and Jiahuan Pei. 2024. MELoRA: Mini-ensemble low-rank adapters for parameter-efficient fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1… find
  55. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  56. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. find
  57. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… find
  58. Pratyusha Sharma, Jordan T. Ash, and Dipendra Misra. 2024. The truth is in there: Improving reasoning in language models with layer-selective rank reduction. In Proceedings of the 12th International Conference on Learning Representations. find
  59. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–…
  60. Dilara Soylu, Christopher Potts, and Omar Khattab. 2024. Fine-tuning and prompt optimization: Two great steps that work better together. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 10696–10710. find
  61. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. find
  62. Zichen Tian, Yaoyao Liu, and Qianru Sun. 2025. Meta-learning hyperparameters for parameter efficient fine-tuning. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 23037– 23047. find
  63. Christophe Tribes, Sacha Benarroch-Lelong, Peng Lu, and Ivan Kobyzev. 2023. Hyperparameter optimization for large language model instruction-tuning. arXiv preprint arXiv:2312.00949. — [arXiv:2312.00949](https://arxiv.org/abs/2312.00949) find
  64. Eileen Wang, Hiba Arnaout, Dhita Pratama, Shuo Yang, Dangyang Liu, Jie Yang, Josiah Poon, Jeff Pan, and Caren Han. 2026. Mmcomet: A large-scale multimodal commonsense knowledge graph for contextual reasoning. arXiv preprint arXiv:2603.01055. — [arXiv:2603.01055](https://arxiv.org/abs/2603.01055) find
  65. Hanqing Wang, Zeguan Xiao, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. 2024. Milora: Harnessing minor singular components for parameter-efficient llm finetuning. arXiv preprint arXiv:2406.09044. find
  66. Zhiguo Wang, Wael Hamza, and Radu Florian. 2017. Bilateral multi-perspective matching for natural language sentences. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, page 4144–4150.
  67. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641. find
  68. Jingyu Wei, Yi Su, Kele Xu, Lingbin Zeng, Bo Liu, and Huaimin Wang. 2024. Demonstrative instruction following in multimodal llms via integrating low-rank adaptation with ensemble learning. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 11435–11441. find
  69. Martin Weyssow, Xin Zhou, Kisub Kim, David Lo, and Houari Sahraoui. 2025. Exploring parameter-efficient fine-tuning techniques for code generation with large language models. ACM Transactions on Software Engineering and Methodology, 34(7):1–25. find
  70. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa…
  71. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229–256. find
  72. Bingyang Wu, Ruidong Zhu, Zili Zhang, Peng Sun, Xuanzhe Liu, and Xin Jin. 2024. Dlora: Dynamically orchestrating requests and adapters for lora llm serving. In Proceedings of the 18th USENIX Symposium on Operating Systems Design and Implementation, pages 911–927. find
  73. Mingjie Zhang, Xiaoling Zhou, Yuxiao Luo, Yiyu Liu, Shikun Zhang, and Wei Ye. 2026. Askd: Reinforcement learning-style knowledge distillation with quality-adaptive skewness. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 34781–34789. find
  74. Dawei Xiang, Wenyan Xu, Kexin Chu, Tianqi Ding, Zixu Shen, Yiming Zeng, Jianchang Su, and Wei Zhang. 2025. Promptsculptor: Multi-agent based text-to-image prompt optimization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 774–… find
  75. Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023b. Adaptive budget allocation for parameter-efficient fine-tuning. In Proceedings of the 11th International Conference on Learning Representations. find
  76. ShiLin Xiao, Tianxiang Xu, Canran Xiao, Weihao Luo, Liwei Hou, and Chuangxin Zhao. 2026. Meta-ucf: Unified task-conditioned lora generation for continual learning in large language models. In The Fourteenth International Conference on Learning Representations. find
  77. Zequn Xie. 2026. Conquer: Context-aware representation with query enhancement for text-based person search. arXiv preprint arXiv:2601.18625. — [arXiv:2601.18625](https://arxiv.org/abs/2601.18625) find
  78. Zequn Xie, Xin Liu, Boyun Zhang, Yuxiao Lin, Sihang Cai, and Tao Jin. 2026. Hvd: Human vision-driven video representation learning for text-video retrieval. arXiv preprint arXiv:2601.16155. — [arXiv:2601.16155](https://arxiv.org/abs/2601.16155) find
  79. Zhipei Xu, Xuanyu Zhang, Qing Huang, Xing Zhou, and Jian Zhang. 2025. Avatarshield: Visual reinforcement learning for human-centric synthetic video detection. arXiv preprint arXiv:2505.15173. — [arXiv:2505.15173](https://arxiv.org/abs/2505.15173) find
  80. Renrui Zhang, Jiaming Han, Chris Liu, Aojun Zhou, Pan Lu, Yu Qiao, Hongsheng Li, and Peng Gao. 2024. Llama-adapter: Efficient fine-tuning of large language models with zero-initialized attention. In The Twelfth International Conference on Learning Representations. find
  81. Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao, and Yining Wang. 2025a. Uora: Uniform orthogonal reinitialization adaptation in parameter efficient fine-tuning of large mdels. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volu… find
  82. Yue Zhang, Hehe Fan, Wei Ji, Yongkang Wong, Roger Zimmermann, and Yi Yang. 2025b. Prompt-aware adapter: Learning adaptive visual tokens for multimodal large language models. IEEE Transactions on Artificial Intelligence. find
  83. Shuo Yang, Caren Han, Yihao Ding, Shuhe Wang, and Eduard Hovy. 2026. Tooltree: Efficient llm tool planning via dual-feedback monte carlo tree search and bidirectional pruning. In The Fourteenth International Conference on Learning Representations. find
  84. Jinman Zhao, Xueyan Zhang, Jiaru Li, Jingcheng Niu, Yulan Hu, Erxue Min, and Gerald Penn. 2025. Tiny budgets, big gains: Parameter placement strategy in parameter super-efficient fine-tuning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6315–6333.
  85. Hua Ye, Siyuan Chen, Ziqi Zhong, Canran Xiao, Haoliang Zhang, Yuhan Wu, and Fei Shen. 2026. Seeing through the conflict: Transparent knowledge conflict handling in retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 34423–34431. find
  86. Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. 2024. Opencodeinterpreter: Integrating code generation with execution and refinement. arXiv preprint arXiv:2402.14658. find
  87. Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. Metamath: Bootstrap your own mathematical questions for large language models. In Proceedings of the 12th International Conference on Learning Representations.
  88. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800, Florence, Italy. Association for Computational Linguistics. find
  89. Longteng Zhang, Lin Zhang, Shaohuai Shi, Xiaowen Chu, and Bo Li. 2023a. Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning. arXiv preprint arXiv:2308.03303.
  90. Xiaofan Zheng, Minnan Luo, and Xinghao Wang. 2025a. Unveiling fake news with adversarial arguments generated by multimodal large language models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7862–7869. find
  91. Xiaofan Zheng, Zinan Zeng, Heng Wang, Yuyang Bai, Yuhan Liu, and Minnan Luo. 2025b. From predictions to analyses: Rationale-augmented fake news detection with large vision-language models. In Proceedings of the ACM on Web Conference 2025, page 5364–5375. find
  92. Xiaofan Zheng, Huixuan Zhang, and Xiaojun Wan. 2025c. Tracing training footprints: A calibration approach for membership inference attacks against multimodal large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 17179–17191. find
  93. Hongyun Zhou, Xiangyu Lu, Wang Xu, Conghui Zhu, and Tiejun Zhao. 2025a. Lora-drop: Efficient lora parameter pruning based on output evaluation. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5530–5543.
  94. Xiaoling Zhou, Ou Wu, Weiyao Zhu, and Ziyang Liang. 2022. Understanding difficulty-based sample weighting with a universal difficulty measure. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 68–84. Springer. find
  95. Xiaoling Zhou, Wei Ye, Zhemg Lee, Lei Zou, and Shikun Zhang. 2025b. Valuing training data via causal inference for in-context learning. IEEE Transactions on Knowledge and Data Engineering. find
  96. Xiaoling Zhou, Wei Ye, Yidong Wang, Chaoya Jiang, Zhemg Lee, Rui Xie, and Shikun Zhang. 2024. Enhancing in-context learning via implicit demonstration augmentation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2810– 2828. find
  97. Xiaoling Zhou, Mingjie Zhang, Zhemg Lee, Yuncheng Hua, Wei Ye, Flora D Salim, Shikun Zhang, and 1 others. 2025c. Boosting resilience of large language models through causality-driven robust optimization. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  98. Xiaoling Zhou, Mingjie Zhang, Zhemg Lee, Wei Ye, and Shikun Zhang. 2025d. Hademif: Hallucination detection and mitigation in large language models. In The Thirteenth International Conference on Learning Representations. find
  99. Bojia Zi, Xianbiao Qi, Lingzhi Wang, Jianan Wang, Kam-Fai Wong, and Lei Zhang. 2023. Delta-lora: Fine-tuning high-rank parameters with the delta of low-rank matrices. arXiv preprint arXiv:2309.02411. — [arXiv:2309.02411](https://arxiv.org/abs/2309.02411) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2026-lelora-learnable-low-rank-adaptation/