Elamur Research⌕
☰
home / papers / 2026 / GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models

GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models

Paper · 2026 · Zhiwen Ruan, Yichao Du, Jianjie Zheng et al.

Summary

Adapting instruction-tuned LLMs to a specific domain is risky: fine-tuning them directly can disturb the balance of skills built during post-training. A common workaround trains an adapter on the pretrained base model and then merges it into the instruct model, as in Shadow-FT and Chat Vector. In those methods, however, the instruct model plays no part until the final merge. GIFT gives the instruct model a role during training. It runs one offline forward pass of the instruct model over the training data and records how probable each reference token is. These token-level confidence scores, used without normalization, then weight the cross-entropy loss when a LoRA adapter is trained on the base model. The adapter is finally merged into the instruct model with standard LoRA merging. As a result, tokens the instruct model is unsure about contribute little to learning. The authors test GIFT on math and medical QA across several model families and sizes. It beats direct fine-tuning and merge-based baselines while keeping general instruction-following intact. This makes it a cheap and practical recipe for specializing aligned models.

Key points

  • On five math benchmarks (Math500, Minerva Math, OlympiadBench, AIME 2024, AMC 2023), GIFT raises accuracy by 5.2% on average over Llama3.1-8B-Instruct. On medical QA, the gain is 6.2%.
  • The scaling study covers the Qwen2.5 family from 0.5B to 32B. GIFT outperforms both the instruct baseline and Shadow-FT at every size, and its advantage holds even for the 14B and 32B models.
  • An analysis of more than 500k NuminaMath-CoT tokens with Llama3.1-8B shows how the learning signal shifts:
    • Low-confidence tokens fall from 79.7% of the signal under Shadow-FT to 29.6% under GIFT.
    • High-confidence tokens rise from 6.9% to 31.5%.
  • Offline annotation is cheap. Scoring 2,000 NuminaMath-CoT samples took about 2 minutes on a single RTX 4090, used under 22GB of memory, and produced a file of roughly 12MB.
  • The code is available at github.com/sustech-nlp/gift. According to the authors, GIFT is complementary to merging methods such as TIES and DARE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🏢Alibaba GroupHangzhou · China168 works · 15,386 citations🎓Xiamen UniversityXiamen · China21 works · 2,258 citations🎓Southern University of Science and TechnologyShenzhen · China19 works · 1,337 citations🎓Shanghai University of Finance and EconomicsShanghai · China13 works · 825 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Zhiwen Ruan, Yichao Du, Jianjie Zheng, Longyue Wang, Yun Chen, Peng Li, Jinsong Su, Yang Liu and Guanhua Chen. 2026. GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.358

@inproceedings{ruan2026gift,
  title = {{GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models}},
  author = {Zhiwen Ruan, Yichao Du, Jianjie Zheng, Longyue Wang, Yun Chen, Peng Li, Jinsong Su, Yang Liu and Guanhua Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.358},
  url = {https://doi.org/10.18653/v1/2026.acl-long.358}
}

References

45 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. AI@Meta. 2024. Llama 3 model card. find
  3. American Institute of Mathematics. 2024. find
  4. Aime 2024 competition mathematical problems. find
  5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  6. Sheng Cao, Mingrui Wu, Karthik Prasad, Yuandong Tian, and Zechun Liu. 2025. Param$\\delta$ for direct mixing: Post-train large language model at zero cost. In The Thirteenth International Conference on Learning Representations. find
  7. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL‘05), pages 173–180, Ann Arbor, Michigan. Association for Computational Linguistics.
  8. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, and etc. 2021. Evaluating large language models trained on code. Preprint, arXiv:2107.03374.
  9. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1–39. find
  10. William Fleshman and Benjamin Van Durme. 2024. Re-adapt: Reverse engineered adaptation of large language models. Preprint, arXiv:2405.15007. — [arXiv:2405.15007](https://arxiv.org/abs/2405.15007) find
  11. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the…
  12. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300.
  13. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
  14. Shih-Cheng Huang, Pin-Zu Li, Yu-chi Hsu, Kuang-Ming Chen, Yu Tung Lin, Shih-Kai Hsiao, Richard Tsai, and Hung-yi Lee. 2024. Chat vector: A simple approach to equip LLMs with instruction following and model alignment in new languages. In Proceedings of the 62nd Annual Meeting of the Association for C… find
  15. Weiyang Huang, Xuefeng Bai, Kehai Chen, Xinyang Chen, Yibin Chen, Weili Guan, and Min Zhang. 2026. Sat: Balancing reasoning accuracy and efficiency with stepwise adaptive thinking. Preprint, arXiv:2604.07922.
  16. Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421.
  17. Peng Lai, Jianjie Zheng, Sijie Cheng, Yun Chen, Peng Li, Yang Liu, and Guanhua Chen. Beyond the surface: Enhancing llm-as-a-judge alignment with human via internal representations. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  18. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022. Solving quantitative reasoning problems with language models. Advances in neural information processing systems, 35:3843– 3857.
  19. Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. 2024. Numinamath. [https://huggingface.co/ AI-MO/NuminaMath-CoT](https://gi… find
  20. Pin-Jie Lin, Rishab Balasubramanian, Fengyuan Liu, Nikhil Kandpal, and Tu Vu. 2025. Efficient model development through fine-tuning transfer. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 2617–2636, Suzhou, China. Association for Computational Lingu… find
  21. Mathematical Association of America. 2023. find
  22. Amc 2023 competition problems. find
  23. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pages 248–260. PMLR.
  24. Zhiwen Ruan, Yun Chen, Yutao Hou, Peng Li, Yang Liu, and Guanhua Chen. 2025a. Unveiling over-memorization in finetuning llms for reasoning tasks. Preprint, arXiv:2508.04117. — [arXiv:2508.04117](https://arxiv.org/abs/2508.04117) find
  25. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations.
  26. Zhiwen Ruan, Yixia Li, Yefeng Liu, Yun Chen, Weihua Luo, Peng Li, Yang Liu, and Guanhua Chen. 2025b. G2: Guided generation for enhanced output diversity in LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 14116–14134, Suzhou, China. Association f…
  27. Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. 2019. Averaging weights leads to wider optima and better generalization. Preprint, arXiv:1803.05407.
  28. Zhiwen Ruan, Yixia Li, He Zhu, Yun Chen, Peng Li, Yang Liu, and Guanhua Chen. 2025c. Enhancing large language model reasoning via selective critical token fine-tuning. Preprint, arXiv:2510.10974. — [arXiv:2510.10974](https://arxiv.org/abs/2510.10974) find
  29. Zhiwen Ruan, Yixia Li, He Zhu, Longyue Wang, Weihua Luo, Kaifu Zhang, Yun Chen, and Guanhua Chen. 2025d. LayAlign: Enhancing multilingual reasoning in large language models via layer-wise adaptive fusion and alignment strategy. In Findings of the Association for Computational Linguistics: NAACL 2025… find
  30. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  31. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling llm test-time compute optimally can be more effective than scaling model parameters. International Conference on Learning Representations (ICLR 2025).
  32. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems, volume 33, pages 3008–3021. Curran Associates, Inc. find
  33. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  34. Hanqing Wang, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. 2025. MiLoRA: Harnessing minor singular components for parameter-efficient LLM finetuning. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Tech… find
  35. Tianyi Wang, Yixia Li, Long Li, Yibiao Chen, Shaohan Huang, Yun Chen, Peng Li, Yang Liu, and Guanhua Chen. 2026. Sppo: Sequence-level ppo for long-horizon reasoning tasks. Preprint, arXiv:2604.08865.
  36. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022. Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks. In EMNLP. find
  37. Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui. 2024. From decoding to meta-generation: Inference-time algorithms for large language models. Transactions on Machine Learning Research. Survey Certification.
  38. Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing in…
  39. Taiqiang Wu, Runming Yang, Jiayi Li, Pengfei Hu, Yik-Chung Wu, Ngai Wong, and Yujiu Yang. 2025a. Shadow-ft: Tuning instruct model via training on paired base model. Preprint, arXiv:2505.12716. — [arXiv:2505.12716](https://arxiv.org/abs/2505.12716) find
  40. Yongliang Wu, Yizhou Zhou, Zhou Ziheng, Yingzhe Peng, Xinyu Ye, Xinting Hu, Wenbo Zhu, Lu Qi, Ming-Hsuan Yang, and Xu Yang. 2025b. On the generalization of sft: A reinforcement learning perspective with reward rectification. Preprint, arXiv:2508.05629. — [arXiv:2508.05629](https://arxiv.org/abs/2508.05629) find
  41. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. Ties-merging: Resolving interference when merging models. Preprint, arXiv:2306.01708.
  42. Jian Yang, Wei Zhang, Jiajun Wu, Junhang Cheng, Tuney Zheng, Fanglin Xu, Weicheng Gu, Lin Jing, Yaxin Du, Joseph Li, et al. 2026. Incoder-32b-thinking: Industrial code world model for thinking. arXiv preprint arXiv:2604.03144. find
  43. Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. Preprint, arXiv:2311.03099.
  44. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911.
  45. He Zhu, Junyou Su, Peng Lai, Ren Ma, Wenjia Zhang, Linyi Yang, and Guanhua Chen. 2026. Anchored supervised fine-tuning. In The Fourteenth International Conference on Learning Representations. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ruan-2026-gift-guided-fine-tuning-transfer/