Elamur Research⌕
☰
home / papers / 2026 / On the Representation Geometry of LoRA Model Merging

On the Representation Geometry of LoRA Model Merging

Paper · 2026 · Chenyang Lu, Jiaru Li, Jinman Zhao et al.

Summary

Combining several task-specific LoRA adapters into one model without further training is hard because their low-rank updates tend to occupy misaligned subspaces. Leading training-free approaches such as KnOTS align updates through a shared SVD basis, which assumes that a single global subspace is enough. This paper tests that assumption by examining the geometry of LoRA weight updates, using the adapters released with KnOTS. Subspace-overlap and energy analyses show that SVD bases and column-selection CUR bases (CUR keeps the columns with the largest ℓ2 norms) capture different directions. SVD favours structure common to all tasks, while CUR keeps localized, task-specific components. The authors build on this trade-off with a training-free merging procedure that blends SVD-aligned and CUR-aligned merged updates. The method outperforms existing gradient-free LoRA merging baselines on ViT-B/32 across eight image classification datasets and on Llama-3 8B across NLI tasks, including a harder joint-task evaluation. The study's broader lesson is that merging quality depends on the representation basis, not only on the rule used to combine updates.

Key points

  • Core finding: SVD and CUR bases are systematically misaligned at the subspace level and have complementary strengths, which exposes a trade-off between preserving shared structure and preserving task-specific detail.
  • Language results: On Llama-3 8B NLI tasks, the method reaches the best average normalized accuracy among compared methods, at 93.5%.
  • Vision results: Experiments use ViT-B/32 on Cars, DTD, EuroSAT, GTSRB, MNIST, RESISC45, SUN397 and SVHN.
    • In the joint-task setting, the eight label sets are merged into 748 unique classes.
    • The method leads all baselines on every Hits@k metric.
  • Ablations:
    • Interpolating between pure CUR (α=0) and pure SVD (α=1) gives a unimodal curve that peaks near α≈0.6.
    • Merging in the CUR "UR" space beats merging in the reduced "R" space, especially on MNIST and SVHN.
    • Adding LoRA-LEGO did not bring consistent further gains.
  • Code and limitations:
    • Code is available at github.com/lcytoronto/knots_cur.
    • The analysis covers weight space only and supervised fine-tuning only; whether the findings hold for other adaptation paradigms, such as reinforcement learning, is untested.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of OxfordOxford · United Kingdom104 works · 157,183 citations🎓University of TorontoToronto · Canada96 works · 142,007 citations🏢BaiduBeijing · China74 works · 15,263 citations🎓Northwestern UniversityEvanston · United States41 works · 2,194 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Latent semantic indexing · 29 works

Cite

Cite as Chenyang Lu, Jiaru Li, Jinman Zhao, Xinran Chen, Yining Wang, Renyi Cai, Yuchen Li and Chao He. 2026. On the Representation Geometry of LoRA Model Merging. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.261

@inproceedings{lu2026representation,
  title = {{On the Representation Geometry of LoRA Model Merging}},
  author = {Chenyang Lu, Jiaru Li, Jinman Zhao, Xinran Chen, Yining Wang, Renyi Cai, Yuchen Li and Chao He},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.261},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.261}
}

References

74 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computational Linguistics. find
  2. Gong Cheng, Junwei Han, and Xiaoqiang Lu. 2017. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865–1883. find
  3. Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. 2014. Describing textures in the wild. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 3606–3613. find
  4. Junhao Dong, Raoof Zare Moayedi, Yew-Soon Ong, and Seyed-Mohsen Moosavi-Dezfooli. 2026a. Allies teach better than enemies: Inverse adversaries for robust knowledge distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence. find
  5. Junhao Dong, Cong Zhang, Xinghua Qu, Sua Qi Rong, Nguyen Duc Thai, Wenbo Pan, Xinfeng Li, Tongliang Liu, Piotr Koniusz, and Yew-Soon Ong. 2026b. Tug-of-war no more: Harmonizing accuracy and robustness in vision-language models via stability-aware task vector merging. In The Fourteenth International… find
  6. Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024. LoRAMoE: Alleviating world knowledge forgetting in large language models via MoE-style plugin. In Proce…
  7. Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. 2018. Essentially no barriers in neural network energy landscape. In International conference on machine learning, pages 1309–1318. PMLR. find
  8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  9. Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2022. BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1–9, Dublin, Ireland. Asso…
  10. Yuchun Fan, Yongyu Mu, YiLin Wang, Lei Huang, Junhao Ruan, Bei Li, Tong Xiao, Shujian Huang, Xiaocheng Feng, and Jingbo Zhu. 2025a. SLAM: Towards efficient multilingual reasoning via selective language alignment. In Proceedings of the 31st International Conference on Computational Linguistics, pages…
  11. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference.
  12. Yuchun Fan, Yilin Wang, Yongyu Mu, Lei Huang, Bei Li, Xiaocheng Feng, Tong Xiao, and JingBo Zhu. 2025b. Language-specific layer matters: Efficient multilingual enhancement for large vision-language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 12473– 12500,… find
  13. Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. 2018. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances in neural information processing systems, 31. find
  14. Gene H Golub and Charles F Van Loan. 2013. Matrix computations. JHU press. find
  15. Yunchuan Guan, Yu Liu, Ke Zhou, Hui Li, Sen Jia, Zhiqi Shen, Ziyang Wang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, and Lei Li. 2025. Learning an efficient optimizer via hybrid-policy sub-trajectory balance. Preprint, arXiv:2511.00543. — [arXiv:2511.00543](https://arxiv.org/abs/2511.00543) find
  16. Farhadi. 2023a. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations.
  17. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023b. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations.
  18. Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. 2023. Dataless knowledge fusion by merging weights of language models. In The Eleventh International Conference on Learning Representations. find
  19. Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. Scitail: A textual entailment dataset from science question answering. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). find
  20. Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. Preprint, arXiv:2403.14608.
  21. Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3d object representations for fine-grained categorization. In 2013 IEEE International Conference on Computer Vision Workshops, pages 554–561. find
  22. Soufiane Hayou, Nikhil Ghosh, and Bin Yu. 2024. LoRA+: Efficient low rank adaptation of large models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 17783– 17806. PMLR. find
  23. Yann LeCun. 1998. The mnist database of handwritten digits. http://yann.lecun. com/exdb/mnist/. ↗
  24. Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2018. Introducing eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. In IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, pages 204–207. find
  25. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Ma… find
  26. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  27. Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee. 2023. LLM-adapters: An adapter family for parameter-efficient fine-tuning of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,…
  28. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processi…
  29. Kaiyang Li, Shaobo Han, Qing Su, Wei Li, Zhipeng Cai, and Shihao Ji. 2025a. Uni-loRA: One vector is all you need. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  30. Wenhao Li, Yuxin Zhang, Gen Luo, Haiyuan Wan, Ziyang Gong, Fei Chao, and Rongrong Ji. 2025b. Spotlight attention: Towards efficient llm generation via non-linear hashing-based kv cache retrieval. Preprint, arXiv:2508.19740. find
  31. Wenhao Li, Yuxin Zhang, Gen Luo, Daohai Yu, and Rongrong Ji. 2025c. Training long-context LLMs efficiently via chunk-wise optimization. In ACL 2025 findings. find
  32. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… find
  33. Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61–6…
  34. Zeyu Liu, Souvik Kundu, Anni Li, Junrui Wan, Lianghao Jiang, and Peter Beerel. 2024. AFLoRA: Adaptive freezing of low rank adaptation in parameter efficient fine-tuning of large models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers… find
  35. Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu. 2024. Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models. Preprint, arXiv:2402.12851. — [arXiv:2402.12851](https://arxiv.org/abs/2402.12851) find
  36. Yulong Mao, Kaiyu Huang, Changhao Guan, Ganglin Bao, Fengran Mo, and Jinan Xu. 2024. DoRA: Enhancing parameter-efficient fine-tuning with dynamic rank distribution. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11662– 11675,… find
  37. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages…
  38. Ryota Miyano and Yuki Arase. 2025. Adaptive LoRA merge with parameter pruning for low-resource generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19353–19366, Vienna, Austria. Association for Computational Linguistics. find
  39. Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, and 1 others. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, page 7. Granada. find
  40. Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. 2023. Task arithmetic in the tangent space: Improved editing of pre-trained models. Advances in Neural Information Processing Systems, 36:66727–66754. find
  41. Aniello Panariello, Daniel Marczak, Simone Magistri, Angelo Porrello, Bartłomiej Twardowski, Andrew D Bagdanov, Simone Calderara, and Joost van de Weijer. Accurate and efficient low-rank model merging in core space. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  42. Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, and Samy Jelassi. 2025. LoRA soups: Merging LoRAs for practical skill composition tasks. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 644–655, Abu Dhabi, UAE. Assoc… find
  43. Peijun Qing, Chongyang Gao, Yefan Zhou, Xingjian Diao, Yaoqing Yang, and Soroush Vosoughi. 2024. AlphaLoRA: Assigning LoRA experts based on layer training quality. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 20511–20523, Miami, Florida, USA. Assoc… find
  44. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8…
  45. Pengjie Ren, Chengshun Shi, Shiguang Wu, Mengqi Zhang, Zhaochun Ren, Maarten de Rijke, Zhumin Chen, and Jiahuan Pei. 2024. MELoRA: Mini-ensemble low-rank adapters for parameter-efficient fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1… find
  46. Jingzhe Shi, Qinwei Ma, Hongyi Liu, Hang Zhao, Jenq-Neng Hwang, and Lei Li. 2026. Intrinsic entropy of context length scaling in LLMs. In The Fourteenth International Conference on Learning Representations. find
  47. Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. 2011. The german traffic sign recognition benchmark: A multi-class classification competition. In The 2011 International Joint Conference on Neural Networks, pages 1453–1460. find
  48. George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman. 2025. Model merging with SVD to tie the knots. In The Thirteenth International Conference on Learning Representations. find
  49. Dennis Tang, Prateek Yadav, Yi-Lin Sung, Jaehong Yoon, and Mohit Bansal. 2025. LoRA merging with SVD: Understanding interference and preserving performance. In ICML 2025 Workshop on Reliable and Responsible Foundation Models. find
  50. Chunlin Tian, Zhan Shi, Zhijiang Guo, Li Li, and Cheng zhong Xu. 2024. HydraloRA: An asymmetric loRA architecture for efficient fine-tuning. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. find
  51. Mojtaba Valipour, Mehdi Rezagholizadeh, Ivan Kobyzev, and Ali Ghodsi. 2023. DyLoRA: Parameter-efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 32…
  52. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations.
  53. Haoyu Wang, Tianci Liu, Ruirui Li, Monica Xiao Cheng, Tuo Zhao, and Jing Gao. 2024. RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,… find
  54. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa…
  55. Hui Wu, Haoquan Zhai, Yuchen Li, Hengyi Cai, Peirong Zhang, Yidan Zhang, Lei Wang, Chunle Wang, Yingyan Hou, Shuaiqiang Wang, and Dawei Yin. 2025. Mara: A multimodal adaptive retrieval-augmented framework for document question answering. In Proceedings of the 33rd ACM International Conference on Mul… find
  56. Jianxiong Xiao, Krista A Ehinger, James Hays, Antonio Torralba, and Aude Oliva. 2016. Sun database: Exploring a large collection of scene categories. International Journal of Computer Vision, 119(1):3–22. find
  57. Xi Xiao, Chenrui Ma, Yunbei Zhang, Chen Liu, Zhux-uanzi Wang, Yanshu Li, Lin Zhao, Guosheng Hu, Tianyang Wang, and Hao Xu. 2026a. Not all directions matter: Toward structured and task-aware low-rank adaptation. Preprint, arXiv:2603.14228. — [arXiv:2603.14228](https://arxiv.org/abs/2603.14228) find
  58. Xi Xiao, Yunbei Zhang, Lin Zhao, Yiyang Liu, Xiaoying Liao, Zheda Mai, Xingjian Li, Xiao Wang, Hao Xu, Jihun Hamm, Xue Lin, Min Xu, Qifan Wang, Tianyang Wang, and Cheng Han. 2026b. Prompt-based adaptation in large-scale vision models: A survey. Preprint, arXiv:2510.13219. — [arXiv:2510.13219](https://arxiv.org/abs/2510.13219) find
  59. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. TIES-merging: Resolving interference when merging models. In Thirty-seventh Conference on Neural Information Processing Systems.
  60. Yutong Yin and Zhaoran Wang. 2025. Are transformers able to reason by connecting separated knowledge in training data? In The Thirteenth International Conference on Learning Representations. find
  61. Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: absorbing abilities from homologous models as a free lunch. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
  62. Lei Yu, Jingcheng Niu, Zining Zhu, Xi Chen, and Gerald Penn. 2025. Sheaf discovery with joint computation graph pruning and flexible granularity. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 8822–8837, Suzhou, China. Association for Computational L… find
  63. Juzheng Zhang, Jiacheng You, Ashwinee Panda, and Tom Goldstein. 2025a. LoRI: Reducing cross-task interference in multi-task low-rank adaptation. In Second Conference on Language Modeling. find
  64. Longteng Zhang, Lin Zhang, Shaohuai Shi, Xiaowen Chu, and Bo Li. 2023a. Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning. Preprint, arXiv:2308.03303.
  65. Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023b. Adaptive budget allocation for parameter-efficient fine-tuning. In The Eleventh International Conference on Learning Representations. find
  66. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. 2025b. A survey on test-time scaling in large language models: What, how, where, and how well? Preprint, arXiv:2503.24235. find
  67. Tong Zhang, Kuofeng Gao, Jiawang Bai, Leo Yu Zhang, Xin Yin, Zonghui Wang, Shouling Ji, and Wenzhi Chen. 2025c. Pre-training CLIP against data poisoning with optimal transport-based matching and alignment. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pag… find
  68. Xueyan Zhang, Jinman Zhao, Zhifei Yang, Yibo Zhong, Shuhao Guan, Linbo Cao, and Yining Wang. 2025d. UORA: Uniform orthogonal reinitialization adaptation in parameter efficient fine-tuning of large models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol…
  69. Jinman Zhao, Erxue Min, Hui Wu, Ziheng Li, Zexu Sun, Hengyi Cai, Shuaiqiang Wang, Xu Chen, and Gerald Penn. 2026. Beyond step pruning: Information theory based step-level optimization for self-refining large language models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41):34941… find
  70. Jinman Zhao, Xueyan Zhang, Jiaru Li, Jingcheng Niu, Yulan Hu, Erxue Min, and Gerald Penn. 2025a. Tiny budgets, big gains: Parameter placement strategy in parameter super-efficient fine-tuning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6315–6333,…
  71. Ziyu Zhao, Tao Shen, Didi Zhu, Zexi Li, Jing Su, Xuwu Wang, and Fei Wu. 2025b. Merging loRAs like playing LEGO: Pushing the modularity of loRA to extremes through rank-wise clustering. In The Thirteenth International Conference on Learning Representations. find
  72. Yibo Zhong and Yao Zhou. 2024. Pear: Pruning and sharing adapters in visual parameter-efficient fine-tuning. Preprint, arXiv:2409.19733. — [arXiv:2409.19733](https://arxiv.org/abs/2409.19733) find
  73. Xiaoling Zhou, Ou Wu, and Nan Yang. 2024a. Adversarial training with anti-adversaries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10210–10227. find
  74. Xiaoling Zhou, Wei Ye, Zhemg Lee, Rui Xie, and Shikun Zhang. 2024b. Boosting model resilience via implicit adversarial data augmentation. Preprint, arXiv:2404.16307. — [arXiv:2404.16307](https://arxiv.org/abs/2404.16307) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lu-2026-representation-geometry-lora-model-merging/