Elamur Research⌕
☰
home / papers / 2026 / Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment

Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment

Paper · 2026 · Yuming Yang, Mingyoung Lai, Wanxu Zhao et al.

Summary

Distilling reasoning from a teacher LLM into a student through SFT on long chain-of-thought (CoT) traces does not reliably improve with teacher strength. What counts is how well the data suits the specific student. Current suitability scores rely on student likelihood, so they favour traces the student could nearly produce itself, and those carry little new signal. The authors trained each of 5 base students on outputs from each of 11 reasoning teachers, giving 55 pairings, using solutions to 5,000 math problems. They then propose the Rank-Surprisal Ratio (RSR): the average token rank under the student divided by the average surprisal (negative log-likelihood). Useful traces have low overall probability but are built from tokens the student still ranks near the top, so a lower RSR means better "informative alignment." RSR needs one forward pass and no verifier or test data. It closely tracks post-SFT accuracy and beats likelihood-based alternatives. It also improves both per-problem trajectory selection and teacher selection, giving a cheap way to match distillation data to a target model.

Key points

  • Teachers range from 4B to 671B parameters, including Deepseek-R1, Qwen-3-235B-Thinking, GPT-OSS-120B/20B, QwQ-32B, Nemotron-Super, Magistral-Small and Phi-4-Reasoning-Plus. Students are base models: Qwen-3-14B, LLaMA-3.1-8B, Qwen-2.5-7B, Qwen-3-4B and Qwen-2.5-3B.
  • Evaluation uses average Acc@4 on AIME'25, AIME'24, AMC'23 and MATH500. Each pairing was trained on three independent generation runs, and the results were averaged.
  • Larger teachers often lose to smaller ones. For example, QwQ-32B frequently outperforms the 671B and 235B teachers. Teachers from distant model families, such as GPT-OSS, tend to transfer worse.
  • Across all students, RSR reaches an average Spearman correlation of 0.86 with post-training performance and consistently outperforms other metrics.
  • The authors note several limitations: the work focuses mainly on math, with some extra checks on GPQA, and the metric lacks a theoretical grounding. Code and data are released at github.com/UmeanNever/RankSurprisalRatio.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fudan UniversityShanghai · China167 works · 20,987 citations🎓University of TorontoToronto · Canada96 works · 142,007 citations🎓The University of SydneySydney · Australia45 works · 68,350 citations🔬Shanghai Artificial Intelligence LaboratoryShanghai · China117 works · 11,281 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Yuming Yang, Mingyoung Lai, Wanxu Zhao, Xiaoran Fan, Zhiheng Xi, Mingqi Wu, Chiyue Huang, Jun Zhao, Haijun Lv, Jian Tong, Yunhua Zhou, Yicheng Zou, Qipeng Guo, Tao Gui, Qi Zhang and Xuanjing Huang. 2026. Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1950

@inproceedings{yang2026which,
  title = {{Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment}},
  author = {Yuming Yang, Mingyoung Lai, Wanxu Zhao, Xiaoran Fan, Zhiheng Xi, Mingqi Wu, Chiyue Huang, Jun Zhao, Haijun Lv, Jian Tong, Yunhua Zhou, Yicheng Zou, Qipeng Guo, Tao Gui, Qi Zhang and Xuanjing Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1950},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1950}
}

References

57 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, and 1 others. 2025. Phi-4-reasoning technical report. arXiv preprint arXiv:2504.21318. find
  2. Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stańczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. 2023. On-policy distillation of language models: Learning from self-generated mistakes. In International Conference on Learning Representations.
  3. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  4. Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, and 1 others. 2025. Llama-nemotron: Efficient reasoning models. arXiv preprint arXiv:2505.00949.
  5. Howard Chen, Noam Razin, Karthik Narasimhan, and Danqi Chen. 2025a. Retaining by doing: The role of on-policy data in mitigating forgetting. CoRR, abs/2510.18874. find
  6. Xinghao Chen, Zhijing Sun, Wenjin Guo, Miaoran Zhang, Yanjun Chen, Yirong Sun, Hui Su, Yijie Pan, Dietrich Klakow, Wenjie Li, and Xiaoyu Shen. 2025b. Unveiling the key factors for distilling chain-of-thought reasoning. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, A… find
  7. Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009. Pearson correlation coefficient. Noise reduction in speech processing, pages 1–4. find
  8. Tri Dao. 2024. FlashAttention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations (ICLR).
  9. Dongyi Ding, Tiannan Wang, Chenghao Zhu, Meiling Tao, Yuchen Eleanor Jiang, and Wangchunshu Zhou. 2025. Micota: Bridging the learnability gap with intermediate cot and teacher assistants. ArXiv, abs/2507.01887. — [arXiv:2507.01887](https://arxiv.org/abs/2507.01887) find
  10. Lishui Fan, Yu Zhang, Mouxiang Chen, and Zhongxin Liu. 2025. Posterior-grpo: Rewarding reasoning processes in code generation. CoRR, abs/2508.05170. — [arXiv:2508.05170](https://arxiv.org/abs/2508.05170) find
  11. Roger B. Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamile Lukosiute, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R. Bowman. 2023. Studying large language model general… find
  12. Etash Kumar Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, and 31 others. 2025.… — OpenThoughts: Data Recipes for Reasoning Models
  13. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Deepseek-r1 incentivizes reasoning in llms through rein… find
  14. John Hale. 2001. A probabilistic earley parser as a psycholinguistic model. In Language Technologies 2001: The Second Meeting of the North American Chapter of the Association for Computational Linguistics, NAACL 2001, Pittsburgh, PA, USA, June 2-7, 2001. The Association for Computational Linguistics…
  15. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874.
  16. Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. CoRR, abs/1503.02531.
  17. Prateek Humane, Paolo Cudrano, Daniel Z. Kaplan, Matteo Matteucci, Supriyo Chakraborty, and Irina Rish. 2025. Influence functions for efficient data selection in reasoning. CoRR, abs/2510.06108. — [arXiv:2510.06108](https://arxiv.org/abs/2510.06108) find
  18. Gangwei Jiang, Yahui Liu, Zhaoyi Li, Qi Wang, Fuzheng Zhang, Linqi Song, Ying Wei, and Defu Lian. 2025. What makes a good reasoning chain? uncovering structural patterns in long chain-of-thought reasoning. CoRR, abs/2505.22148.
  19. Hoang Anh Just, Myeongseob Ko, and Ruoxi Jia. 2025. Distilling reasoning into student llms: Local naturalness for selecting teacher data. CoRR, abs/2510.03988. — [arXiv:2510.03988](https://arxiv.org/abs/2510.03988) find
  20. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
  21. Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G. Patil, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. 2025a. Llms can easily learn to reason from demonstrations structure, not content, is what matters! CoRR, abs/2502.07374. find
  22. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, and 1 others. 2024. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9):… find
  23. Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubramanian, and Radha Poovendran. 2025b. Small models struggle to learn from strong reasoners. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 202… find
  24. Jinrui Liu, Jeff Wu, Xuanguang Pan, Gavin Cheung, Shuai Ma, and Chongyang Tao. 2025. Air: Post-training data selection for reasoning via attention head influence. arXiv preprint arXiv:2512.13279. — [arXiv:2512.13279](https://arxiv.org/abs/2512.13279) find
  25. Renjie Luo, Jiaxi Li, Chen Huang, and Wei Lu. 2025. Through the valley: Path to effective long cot training for small language models. CoRR, abs/2506.07712.
  26. Lu Ma, Hao Liang, Meiyi Qiang, Lexiang Tang, Xiaochen Ma, Zhen Hao Wong, Junbo Niu, Chengyu Shen, Runming He, Bin Cui, and Wentao Zhang. 2025. Learning what reinforcement learning can’t: Interleaved online fine-tuning for hardest questions. CoRR, abs/2506.07527. — [arXiv:2506.07527](https://arxiv.org/abs/2506.07527) find
  27. Nikolay Mikhaylovskiy. 2025. Zipf’s and heaps’ laws for tokens and llm-generated texts. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 15469–15481. find
  28. Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Matsukawa, and Hassan Ghasemzadeh. 2020. Improved knowledge distillation via teacher assistant. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 5191–5198.
  29. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. CoRR, abs/2501.19393.
  30. Abhishek Panigrahi, Bingbin Liu, Sadhika Malladi, Sham M. Kakade, and Surbhi Goel. 2025. In good graces: Principled teacher selection for knowledge distillation. CoRR, abs/2511.02833. — [arXiv:2511.02833](https://arxiv.org/abs/2511.02833) find
  31. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with GPT-4. CoRR, abs/2304.03277.
  32. Jingyu Peng, Maolin Wang, Hengyi Cai, Yuchen Li, Kai Zhang, Shuaiqiang Wang, Dawei Yin, and Xiangyu Zhao. 2025. Adaswitch: Adaptive switching generation for knowledge distillation. ArXiv, abs/2510.07842. — [arXiv:2510.07842](https://arxiv.org/abs/2510.07842) find
  33. Abhilasha Ravichander, Jillian Fisher, Taylor Sorensen, Ximing Lu, Maria Antoniak, Bill Yuchen Lin, Niloofar Mireshghallah, Chandra Bhagavatula, and Yejin Choi. 2025. Information-guided identification of training data imprint in (proprietary) large language models. In Proceedings of the 2025 Confere… find
  34. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2024. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  35. Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. 2026. Self-distillation enables continual learning. CoRR, abs/2601.19897. find
  36. Safal Shrestha, Minwu Kim, Aadim Nepal, Anubhav Shrestha, and Keith Ross. 2025. Warm up before you train: Unlocking general reasoning in resource-constrained settings. CoRR, abs/2505.13718. find
  37. Lev Semenovich Vygotsky and Michael Cole. 1978. Mind in society: Development of higher psychological processes. Harvard university press. find
  38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  39. Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, and Jun Wang. 2025a. Beyond scaling law: A data-efficient distillation framework for reasoning. CoRR, abs/2508.09883. — [arXiv:2508.09883](https://arxiv.org/abs/2508.09883) find
  40. Yongliang Wu, Yizhou Zhou, Zhou Ziheng, Yingzhe Peng, Xinyu Ye, Xinting Hu, Wenbo Zhu, Lu Qi, Ming-Hsuan Yang, and Xu Yang. 2025b. On the generalization of SFT: A reinforcement learning perspective with reward rectification. CoRR, abs/2508.05629. — [arXiv:2508.05629](https://arxiv.org/abs/2508.05629) find
  41. Wenda Xu, Rujun Han, Zifeng Wang, Long T. Le, Dhruv Madeka, Lei Li, William Yang Wang, Rishabh Agarwal, Chen-Yu Lee, and Tomas Pfister. 2025. Speculative knowledge distillation: Bridging the teacher-student gap through interleaved sampling. In The Thirteenth International Conference on Learning Repr…
  42. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 40 others. 2025a. Qwen3 technical report. CoRR, abs/2505.09388.
  43. Cehao Yang, Xueyuan Lin, Chengjin Xu, Xuhui Jiang, Xiaojun Wu, Honghao Liu, Hui Xiong, and Jian Guo. 2025b. Select2reason: Efficient instruction-tuning data selection for long-cot reasoning. CoRR, abs/2505.17266.
  44. Yuming Yang, Yang Nan, Junjie Ye, Shihan Dou, Xiao Wang, Shuo Li, Huijie Lv, Tao Gui, Qi Zhang, and Xuanjing Huang. 2025c. Measuring data diversity for instruction tuning: A systematic analysis and A reliable metric. In Proceedings of the 63rd Annual Meeting of the Association for Computational Ling…
  45. Junjie Ye, Yuming Yang, Yang Nan, Shuo Li, Qi Zhang, Tao Gui, Xuan-Jing Huang, Peng Wang, Zhongchao Shi, and Jianping Fan. 2025a. Analyzing the effects of supervised fine-tuning on model knowledge from token and parameter levels. In Proceedings of the 2025 Conference on Empirical Methods in Natural… find
  46. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. 2025b. LIMO: less is more for reasoning. CoRR, abs/2502.03387.
  47. Qianjin Yu, Keyu Wu, Zihan Chen, Chushu Zhang, Man-lin Mei, Lingjun Huang, Fang Tan, Yongsheng Du, Kunlin Liu, and Yurui Zhu. 2025. Rethinking the generation of high-quality cot data from the perspective of llm-adaptive question difficulty grading. CoRR, abs/2504.11919. — [arXiv:2504.11919](https://arxiv.org/abs/2504.11919) find
  48. Jerrold H Zar. 2005. Spearman rank correlation. Encyclopedia of biostatistics, 7. find
  49. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
  50. Chen Zhang, Qiuchi Li, Dawei Song, Zheyu Ye, Yan Gao, and Yao Hu. 2025a. Towards the law of capacity gap in distilling language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1, 2…
  51. Dylan Zhang, Qirun Dai, and Hao Peng. 2025b. The best instruction-tuning data are those that fit. CoRR, abs/2502.04194. — [arXiv:2502.04194](https://arxiv.org/abs/2502.04194) find
  52. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Zhihan Guo, Yufei Wang, Irwin King, Xue Liu, and Chen Ma. 2025c. What, how, where, and how well? A survey on test-time scaling in large language models. CoRR, abs/2503.24235. find
  53. Xuechen Zhang, Zijian Huang, Yingcong Li, Chenshun Ni, Jiasi Chen, and Samet Oymak. 2025d. BREAD: branched rollouts from expert anchors bridge SFT & RL for reasoning. CoRR, abs/2506.17211. — [arXiv:2506.17211](https://arxiv.org/abs/2506.17211) find
  54. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),…
  55. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. LIMA: less is more for alignment. In Advances in Neural Information Processing Systems 36: Annual Confere… find
  56. George Kingsley Zipf. 1936. The psycho-biology of language: An introduction to dynamic philology. Routledge. find
  57. Jiaru Zou, Ling Yang, Jingwen Gu, Jiahao Qiu, Ke Shen, Jingrui He, and Mengdi Wang. 2025. Reasonflux-prm: Trajectory-aware prms for long chain-of-thought reasoning in llms. CoRR, abs/2506.18896. — [arXiv:2506.18896](https://arxiv.org/abs/2506.18896) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-which-reasoning-trajectories-teach-students/