Elamur Research⌕
☰
home / papers / 2026 / A Layer-wise Analysis of Supervised Fine-Tuning

A Layer-wise Analysis of Supervised Fine-Tuning

Paper · 2026 · Qinghua Zhao, Xueling Gong, Xinyu Chen et al.

Summary

The paper asks where, across a model's depth, supervised fine-tuning (SFT) actually changes an LLM. Standard LoRA spreads its updates evenly over all layers, which assumes every layer matters equally for alignment. The authors compare base and SFT checkpoints of decoder-only models from 1B to 32B parameters (OLMo2 family, Mistral-7B). They use several kinds of measurement:

  • Information-theoretic: matrix-based entropy, effective rank.
  • Geometric: CKA, cosine similarity.
  • Optimization: Frobenius norm of attention weight changes.

They also run layer-wise probing and layer swapping. The results show a clear depth pattern. Layers between roughly 20% and 80% of depth stay stable. The top layers diverge sharply and receive the largest weight updates, which suggests they are where old features get overwritten and forgetting happens. Building on this, the authors propose Mid-Block Efficient Tuning, which places LoRA adapters only in intermediate layers. On GSM8K it beats full-depth LoRA with a similar or smaller parameter budget. The authors frame the method as a proof-of-concept for their analysis rather than a rival PEFT technique. The main takeaway is that alignment is localized in particular layers, so layer choice is a key lever for efficient and forgetting-resistant fine-tuning.

Key points

  • On OLMo2-7B, adapting only the upper-middle segment reached 0.375 accuracy on GSM8K, compared with 0.28 for full-layer LoRA. That is about 10 points; the paper reports "up to 10.2%".
  • Middle-segment configurations also won on larger models:
    • OLMo2-13B: 0.30 vs 0.27.
    • OLMo2-32B: 0.32 vs 0.29, with fewer trainable parameters.
  • Tuning only edge segments performed poorly:
    • Bottom only, OLMo2-13B: 0.22.
    • Top only, OLMo2-1B: 0.135 vs a 0.19 baseline.
  • Setup: layers were split into M=5 segments with LoRA rank 8 or 64, keeping total trainable parameters roughly constant. Ablations with M=3 and M=10 showed an inverted-U pattern, meaning the middle region is a broad plateau rather than a narrow peak.
  • Scope and code: the study covers only dense decoder-only models and only the SFT stage (no RLHF/DPO or MoE). Code is available at github.com/lshowway/base.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Lanzhou UniversityLanzhou · China8 works · 24 citations🎓Hefei UniversityHefei · China4 works · 1 citation

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Qinghua Zhao, Xueling Gong, Xinyu Chen, Zhongfeng Kang and Xinlu Li. 2026. A Layer-wise Analysis of Supervised Fine-Tuning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.453

@inproceedings{zhao2026layer,
  title = {{A Layer-wise Analysis of Supervised Fine-Tuning}},
  author = {Qinghua Zhao, Xueling Gong, Xinyu Chen, Zhongfeng Kang and Xinlu Li},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.453},
  url = {https://doi.org/10.18653/v1/2026.acl-long.453}
}

References

28 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Khai Loong Aw, Syrielle Montariol, Badr AlKhamissi, Martin Schrimpf, and Antoine Bosselut. 2024. Instruction-tuning aligns LLMs to the human brain. In First Conference on Language Modeling. find
  2. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhai… — Evaluating Large Language Models Trained on Code
  3. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tai, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan… find
  4. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems.
  5. Hongzhe Du, Weikai Li, Min Cai, Karim Saraipour, Zimin Zhang, Himabindu Lakkaraju, Yizhou Sun, and Shichang Zhang. 2025. How post-training reshapes llms: A mechanistic view on knowledge, truthfulness, refusal, and confidence. find
  6. Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, and Dinesh Manocha. 2024. A closer look at the limitations of instruction tuning. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Ma… find
  7. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:…
  8. Zirui He, Haiyan Zhao, Yiran Qiao, Fan Yang, Ali Payani, Jing Ma, and Mengnan Du. 2025. Saif: A sparse autoencoder framework for interpreting and steering instruction following of language models. find
  9. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In International Conference on Learning Representations.
  10. Saffron Huang, Divya Siddarth, Liane Lovitt, Thomas I. Liao, Esin Durmus, Alex Tamkin, and Deep Ganguli. 2024. Collective constitutional ai: Aligning a language model with public input. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’24, page 1395–1417… find
  11. Itay Itzhak, Gabriel Stanovsky, Nir Rosenfeld, and Yonatan Belinkov. 2024. Instructed to bias: Instruction-tuned language models exhibit emergent cognitive bias. Transactions of the Association for Computational Linguistics, 12:771–785. find
  12. Aditi Jha, Sam Havens, Jeremy Dohmann, Alex Trott, and Jacob Portes. 2023. Limit: Less is more for instruction tuning across evaluation paradigms. find
  13. Gangwei Jiang, Caigao JIANG, Zhaoyi Li, Siqiao Xue, JUN ZHOU, Linqi Song, Defu Lian, and Ying Wei. 2025. Unlocking the power of function vectors for characterizing and mitigating catastrophic forgetting in continual instruction tuning. In The Thirteenth International Conference on Learning Represent… find
  14. Po-Nien Kung and Nanyun Peng. 2023. Do models really learn to follow instructions? an empirical study of instruction tuning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1317– 1328, Toronto, Canada. Association for Computa… find
  15. Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. 2024. The unlocking spell on base LLMs: Rethinking alignment via in-context learning. In The Twelfth International Conference on Learning Representations.
  16. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In Advances in Neural Information Processing Systems, volume 36, pages 34892–34916. Curran Associates, Inc.
  17. Kaifeng Lyu, Haoyu Zhao, Xinran Gu, Dingli Yu, Anirudh Goyal, and Sanjeev Arora. 2024. Keeping llms aligned after fine-tuning: The crucial role of prompt templates. Advances in Neural Information Processing Systems, 37:118603–118631. find
  18. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 35:17359–17372. find
  19. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture models. In International Conference on Learning Representations.
  20. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, volume 35, pages 2773…
  21. Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Wan Guanglu, Xunliang Cai, and Le Sun. 2024. Learning or self-aligning? rethinking instruction fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6090… find
  22. Yi Ren and Danica J. Sutherland. 2025. Learning dynamics of LLM finetuning. In The Thirteenth International Conference on Learning Representations. find
  23. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–15522, Bangkok, Thaila… find
  24. Zhengyang Tang, Xingxing Zhang, Benyou Wang, and Furu Wei. 2024. Mathscale: scaling instruction tuning for mathematical reasoning. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
  25. Table 1 reports results for M = 3 segments. The middle segment (Segment 2) achieves the highest accuracy (0.4721), outperforming both the first segment (0.4650) and the last segment (0.4550), confirming the inverted-U-shaped trend at a coarse granularity. *Table 1: Performance with M = 3 Segments (O…
  26. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. Finetuned language models are zero-shot learners. In International Conference on Learning Representations.
  27. Xuansheng Wu, Wenlin Yao, Jianshu Chen, Xiaoman Pan, Xiaoyang Wang, Ninghao Liu, and Dong Yu. 2024. From language modeling to instruction following: Understanding the behavior shift in LLMs after instruction tuning. In Proceedings of the 2024 Conference of the North American Chapter of the Associati… find
  28. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2026-layer-wise-analysis-supervised-fine/