Elamur Research⌕
☰
home / papers / 2026 / On the Step Length Confounding in LLM Reasoning Data Selection

On the Step Length Confounding in LLM Reasoning Data Selection

Paper · 2026 · Bing Wang, Rui Miao, Chen Shen et al.

Summary

The paper looks at naturalness-based selection of supervised fine-tuning data for long chain-of-thought reasoning. These methods rank samples by how probable a target LLM finds them. The authors show that on reasoning data, such scores mainly reward samples with longer reasoning steps (more tokens per step) rather than better ones. They call this bias step length confounding. They trace it to the first token of each step, which often marks a branching point and so receives low probability. In long steps, that low-probability token is outweighed by many others, which pushes the average score up. They propose ASLEC in two forms:

  • ASLEC-DROP leaves first-token probabilities out of the average.
  • ASLEC-CASL uses a causal-debiasing linear regression to remove the effect of the first-token ratio.

Experiments cover LIMO-v2 and AceReason-1.1-SFT, four trained LLMs and five benchmarks. Both variants beat the strongest existing naturalness method, Local LP. This matters because a cheap and widely used filtering heuristic turns out to be biased in a systematic way that can be fixed.

Key points

  • Bias found in all baselines. The analysis generated 16,000 responses on LIMO-v2 (800 problems × 20 responses) from QwQ-32B, Qwen3-32B, DeepSeek-R1-Distill-Qwen-32B and gpt-oss-120b, scored with Qwen3-4B-Base. GRACE, Local LP, Min Entropy and Min Perplex all favoured longer-step samples.
  • Accuracy gains. Averaged over models and datasets, ASLEC-DROP improves accuracy by about 6.28% over Local LP and ASLEC-CASL by about 9.08%.
  • Size of the first-token effect. The regression on all data gives a first-token-ratio coefficient γ of −0.680. A 0.05 gap in that ratio then equals roughly a 3.34% per-token probability drop. Data from gpt-oss-120b shows the strongest effect (γ = −1.284).
  • Low cost. ASLEC-DROP adds no compute, and the regression in ASLEC-CASL fits in seconds.
  • Released resources. Code is at github.com/wangbing1416/ASLEC. The multi-source, multi-solution SFT data is on Hugging Face as the msms-cot-sft collection, including 10k AceReason-1.1-SFT problems with about 64 responses each.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China168 works · 15,386 citations🎓Zhejiang UniversityHangzhou · China172 works · 12,988 citations🎓University of MichiganAnn Arbor · United States70 works · 13,693 citations🎓Jilin UniversityChangchun · China25 works · 5,252 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Bing Wang, Rui Miao, Chen Shen, Shaotian Yan, Kaiyuan Liu, Ximing Li, Xiaosong Yuan, Sinan Fan, Jun Zhang and Jieping Ye. 2026. On the Step Length Confounding in LLM Reasoning Data Selection. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.918

@inproceedings{wang2026step,
  title = {{On the Step Length Confounding in LLM Reasoning Data Selection}},
  author = {Bing Wang, Rui Miao, Chen Shen, Shaotian Yan, Kaiyuan Liu, Ximing Li, Xiaosong Yuan, Sinan Fan, Jun Zhang and Jieping Ye},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.918},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.918}
}

References

42 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. CoRR, abs/2508.10925.
  2. Jinhe Bi, Danqi Yan, Yifan Wang, Wenke Huang, Haokun Chen, Guancheng Wan, Mang Ye, Xun Xiao, Hinrich Schütze, Volker Tresp, and Yunpu Ma. 2025. Cot-kinetics: A theoretical modeling assessing LRM reasoning process. CoRR, abs/2505.13408. — [arXiv:2505.13408](https://arxiv.org/abs/2505.13408) find
  3. Yuyan Bu, Liangyu Huo, Yi Jing, and Qing Yang. 2025. Beyond excess and deficiency: Adaptive length bias mitigation in reward models for RLHF. In Findings of the Association for Computational Linguistics: NAACL, pages 3091–3098. find
  4. Hongzhan Chen, Siyue Wu, Xiaojun Quan, Rui Wang, Ming Yan, and Ji Zhang. 2023. MCC-KD: multi-cot consistent knowledge distillation. In Findings of the Association for Computational Linguistics: EMNLP, pages 6805–6820. find
  5. Xiaoshu Chen, Sihang Zhou, Ke Liang, Xiaoyu Sun, and Xinwang Liu. 2025a. Skip-thinking: Chunk-wise chain-of-thought distillation enable smaller language models to reason better and faster. In Conference on Empirical Methods in Natural Language Processing, pages 12153–12168. find
  6. Yang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee, Peng Xu, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025b. Acereason-nemotron: Advancing math and code reasoning through reinforcement learning. CoRR, abs/2505.16400. find
  7. Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025. Reasoning with exploration: An entropy perspective. CoRR, abs/2506.14758. find
  8. Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding. 2025. The entropy mechanism of reinforcement learning for reasoning language models. CoRR, abs/2… find
  9. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. 2025. Deep think with confidence. CoRR, abs/2508.15260. find
  10. Etash Kumar Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, and 1 others. 2025. Openthoughts: Data recipes for reasoning models. CoRR, abs/2506.04178.
  11. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseekr1: Incentivizing reasoning capability in llms via reinforcement learning. CoRR, abs/2501.12948.
  12. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, and 1 others. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. In Annual Meeting of the Association for…
  13. Xingyue Huang, Rishabh, Gregor Franke, Ziyi Yang, Jiamu Bai, Weijie Bai, Jinhe Bi, Zifeng Ding, Yiqun Duan, Chengyu Fan, and 1 others. 2025. Loong: Synthesize long chain-of-thoughts at scale through verifiers. CoRR, abs/2509.03059. — [arXiv:2509.03059](https://arxiv.org/abs/2509.03059) find
  14. Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. 2024. The impact of reasoning step length on large language models. In Findings of the Association for Computational Linguistics, ACL, pages 1830–1842.
  15. Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostafa Patwary, Mohammad Shoeybi, Bryan Catanzaro, and Yejin Choi. 2025. Prismatic synthesis: Gradient-based data diversification boosts generalization in LLM reasoning. CoRR, abs/2505.20161. — [arXiv:2505.20161](https://arxiv.org/abs/2505.20161) find
  16. Hoang Anh Just, Myeongseob Ko, and Ruoxi Jia. 2025. Distilling reasoning into student llms: Local naturalness for selecting teacher data. CoRR, abs/2510.03988. 2023. Scaling data-constrained language models. In Annual Conference on Neural Information Processing Systems. find
  17. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. CoRR, abs/2501.19393.
  18. Zhewei Kang, Xuandong Zhao, and Dawn Song. 2025. Scalable best-of-n selection for large language models via self-certainty. CoRR, abs/2502.18581. find
  19. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In Conference on Language Modeling.
  20. Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai, Xiaobo Xia, Ming-Kun Xie, Dong-Dong Wu, Biao Liu, Yuheng Jia, Xin Geng, Masashi Sugiyama, and Tat-Seng Chua. 2026. Positive-unlabeled reinforcement learning distillation for on-premise small models. CoRR, abs/2601.20687. — [arXiv:2601.20687](https://arxiv.org/abs/2601.20687) find
  21. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu. 2025. Stop overthinking: A survey on efficient reasoning for large language models. Transactions on Machine Learning Research, 2025.
  22. Zhenyu Lei, Zhen Tan, Song Wang, Yaochen Zhu, Zihan Chen, Yushun Dong, and Jundong Li. 2025. Learning from diverse reasoning paths with routing and collaboration. In Conference on Empirical Methods in Natural Language Processing, pages 2832–2845.
  23. Qwen Team. 2025. Qwq-32b: Embracing the power of reinforcement learning. find
  24. Hang Li, Kaiqi Yang, Yucheng Chu, Hui Liu, and Jiliang Tang. 2025a. Exploring solution divergence and its effect on large language model problem solving. CoRR, abs/2509.22480. — [arXiv:2509.22480](https://arxiv.org/abs/2509.22480) find
  25. Yang Li, Youssef Emad, Karthik Padthe, Jack Lanchantin, Weizhe Yuan, Thao Nguyen, Jason Weston, Shang-Wen Li, Dong Wang, Ilia Kulikov, and Xian Li. 2025b. Naturalthoughts: Selecting and distilling reasoning traces for general reasoning tasks. CoRR, abs/2507.01921. — [arXiv:2507.01921](https://arxiv.org/abs/2507.01921) find
  26. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.
  27. Kaiyuan Liu, Shaotian Yan, Rui Miao, Bing Wang, Chen Shen, Jun Zhang, and Jieping Ye. 2026. Where did this sentence come from? tracing provenance in LLM reasoning distillation. In International Conference on Learning Representations. find
  28. Kevin Lu and Thinking Machines Lab. 2025. On-policy distillation. Thinking Machines Lab: Connectionism. find
  29. Yijia Luo, Yulin Song, Xingyao Zhang, Jiaheng Liu, Weixun Wang, Gengru Chen, Wenbo Su, and Bo Zheng. 2025. Deconstructing long chain-of-thought: A structured reasoning optimization framework for long cot distillation. CoRR, abs/2503.16385. find
  30. Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Aleksandra Piktus, Sampo Pyysalo, Thomas Wolf, and Colin A. Raffel. find
  31. Yijun Tian, Shaoyu Chen, Zhichao Xu, Yawei Wang, Jinhe Bi, Peng Han, and Wei Wang. 2025. Reinforcement mid-training. CoRR, abs/2509.24375. — [arXiv:2509.24375](https://arxiv.org/abs/2509.24375) find
  32. Can Udomcharoenchaikit, Wuttikorn Ponwitayarat, Patomporn Payoungkhamdee, Kanruethai Masuk, Weerayut Buaphet, Ekapol Chuangsuwanich, and Sarana Nutanong. 2022. Mitigating spurious correlation in natural language understanding with counterfactual inference. In Conference on Empirical Methods in Natur… find
  33. Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. 2025. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforceme…
  34. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems.
  35. Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, and Jun Wang. 2025. Beyond scaling law: A data-efficient distillation framework for reasoning. CoRR, abs/2508.09883. — [arXiv:2508.09883](https://arxiv.org/abs/2508.09883) find
  36. Shaotian Yan, Kaiyuan Liu, Chen Shen, Bing Wang, Sinan Fan, Jun Zhang, Yue Wu, Zheng Wang, and Jieping Ye. 2026. Distribution-aligned sequence distillation for superior long-cot reasoning. CoRR, abs/2601.09088. find
  37. Xiaosong Yuan, Chen Shen, Shaotian Yan, Kaiyuan Liu, Xiaofeng Zhang, Sinan Fan, Liang Xie, Wenxiao Wang, Renchu Guan, Ying Wang, and ieping Ye. 2026. Differential fine-tuning large language models towards better diverse reasoning abilities. In International Conference on Learning Representations. find
  38. Xiaosong Yuan, Chen Shen, Shaotian Yan, Xiaofeng Zhang, Liang Xie, Wenxiao Wang, Renchu Guan, Ying Wang, and Jieping Ye. 2024. Instance-adaptive zero-shot chain-of-thought prompting. In Advances in Neural Information Processing Systems. find
  39. Dylan Zhang, Qirun Dai, and Hao Peng. 2025. The best instruction-tuning data are those that fit. CoRR, abs/2502.04194. — [arXiv:2502.04194](https://arxiv.org/abs/2502.04194) find
  40. Han Zhao, Haotian Wang, Yiping Peng, Sitong Zhao, Xiaoyu Tian, Shuaiting Chen, Yunjie Ji, and Xiangang Li. 2025. 1.4 million open-source distilled reasoning dataset to empower large language model training. CoRR, abs/2503.19633. — [arXiv:2503.19633](https://arxiv.org/abs/2503.19633) find
  41. Congming Zheng, Jiachen Zhu, Zhuoying Ou, Yuxiang Chen, Kangning Zhang, Rong Shan, Zeyu Zheng, Mengyue Yang, Jianghao Lin, Yong Yu, and Weinan Zhang. 2025. A survey of process reward models: From outcome signals to process supervisions for large language models. CoRR, abs/2510.08049. — [arXiv:2510.08049](https://arxiv.org/abs/2510.08049) find
  42. Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, and Fuzhen Zhuang. 2022. Generalizing to the future: Mitigating entity bias in fake news detection. In International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2120–2125. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-step-length-confounding-llm-reasoning/