Elamur Research⌕
☰
home / papers / 2026 / Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization

Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization

Paper · 2026 · Tian Xueyun, MingHua Ma, Bingbing Xu et al.

Summary

SFT on distilled chain-of-thought data usually relies on rejection sampling, which keeps only trajectories whose final answer is correct. The authors argue that this discards useful supervision and encourages overfitting to domain-specific shortcuts, which hurts out-of-domain (OOD) reasoning. They distilled trajectories with Qwen3-8B from OpenMathReasoning and MMLU, split them by answer correctness, and fine-tuned Qwen2.5 models (3B–32B) and Llama-3.1 8B on balanced positive-only or negative-only sets. Negative-only training gave slightly weaker in-domain results but consistently stronger OOD transfer.

They explain this from three angles:

  • Data: flawed chains contain diverse error patterns.
  • Optimization: their loss falls more slowly, which counters shortcut learning.
  • Inference: the resulting models show higher entropy, which supports exploration.

Building on this, they propose GLOW (Gain-based LOss Weighting). GLOW trains on unfiltered data and gives more weight to samples whose loss drops little between epochs. It improves OOD accuracy and also provides a better starting point for RL. The work suggests that "failed" reasoning traces are a cheap and valuable training resource.

Key points

  • A taxonomy of negative chains identifies 9 error types covering 22 recurring patterns. Training on negatives raises policy entropy by 35.67% at inference.
  • Compared with positive-only SFT, training on math negatives gave average gains of 11.97% on general reasoning (MMLU, MMLU-Pro, BBH) and 4.11% on other reasoning tasks (ACPBench, HeadQA). Training on MMLU negatives gave gains of 1.98% on math and 1.35% on other reasoning.
  • On Qwen2.5-7B, GLOW scored 2.14% higher on average than standard SFT on mixed data, and 5.51% higher OOD than positive-only SFT.
  • As an initialization for GRPO on GSM8K, GLOW lifted Qwen2.5-7B's MMLU score from 72.82% (standard SFT + RL) to 76.47%.
  • Code is released at github.com/Eureka-Maggie/GLOW.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China415 works · 94,028 citations🎓Harbin Institute of TechnologyHarbin · China185 works · 29,140 citations🏛️Chinese Academy of SciencesBeijing · China246 works · 24,952 citations🔬Institute of Computing TechnologyBeijing · China57 works · 3,782 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Tian Xueyun, MingHua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang and Huawei Shen. 2026. Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1370

@inproceedings{xueyun2026learning,
  title = {{Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization}},
  author = {Tian Xueyun, MingHua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang and Huawei Shen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1370},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1370}
}

References

52 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. 2024. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arXiv:2402.00157.
  2. Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2023. Learning from mistakes makes llm better reasoner. arXiv preprint arXiv:2310.20689.
  3. Art of Problem Solving Foundation. 2023. Amc23 — 2023 american mathematics competitions test set. 40 problems drawn from the 2023 AMC 12 contests. find
  4. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41–48.
  5. Olivier Bousquet and André Elisseeff. 2002. Stability and generalization. Journal of machine learning research, 2(Mar):499–526. find
  6. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161.
  7. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  8. Rohan Deb, Kiran Thekumparampil, Kousha Kalantari, Gaurush Hiranandani, Shoham Sabach, and Branislav Kveton. 2025. Fishersft: Data-efficient supervised fine-tuning of language models using information gain. arXiv preprint arXiv:2505.14826. — [arXiv:2505.14826](https://arxiv.org/abs/2505.14826) find
  9. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  10. Xiang Gao and Kamalika Das. 2024. Customizing language model responses with contrastive in-context learning. In Proceedings of the aaai conference on artificial intelligence, volume 38, pages 18039– 18046. find
  11. Sonam Gupta, Yatin Nandwani, Asaf Yehudai, Dinesh Khandelwal, Dinesh Raghu, and Sachindra Joshi. 2025. Selective self-to-supervised fine-tuning for generalization in large language models. arXiv preprint arXiv:2502.08130. find
  12. Shadi Hamdan and Deniz Yuret. 2025. How much do llms learn from negative examples? arXiv preprint arXiv:2503.14391. — [arXiv:2503.14391](https://arxiv.org/abs/2503.14391) find
  13. Seungwook Han, Jyothish Pari, Samuel J Gershman, and Pulkit Agrawal. 2025. General reasoning requires learning to reason from the get-go. arXiv preprint arXiv:2502.19402. — [arXiv:2502.19402](https://arxiv.org/abs/2502.19402) find
  14. Moritz Hardt, Ben Recht, and Yoram Singer. 2016. Train faster, generalize better: Stability of stochastic gradient descent. In International conference on machine learning, pages 1225–1234. PMLR. find
  15. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, and 1 others. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008.
  16. Yancheng He, Shilong Li, Jiaheng Liu, Weixun Wang, Xingyuan Bu, Ge Zhang, Zhongyuan Peng, Zhaoxiang Zhang, Zhicheng Zheng, Wenbo Su, and 1 others. 2025. Can large language models detect errors in long chain-of-thought reasoning? arXiv preprint arXiv:2502.19361.
  17. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021a. Aligning ai with shared human values. Proceedings of the International Conference on Learning Representations (ICLR).
  18. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021b. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR).
  19. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2024. Measuring mathematical problem solving with the math dataset, 2021. arXiv preprint arXiv:2103.03874, 2.
  20. Maggie Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Poovendran, Graham Neubig, and Xiang Yue. 2025. Does math reasoning improve general llm capabilities? understanding transferability of llm reasoning. arXiv preprint arXiv:2507.00432. find
  21. Harsha Kokel, Michael Katz, Kavitha Srinivas, and Shirin Sohrabi. 2025. Acpbench: Reasoning about action, change, and planning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26559–26568. find
  22. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, and 1 others. 2022. Solving quantitative reasoning problems with language models. Advances in neural information processing systems, 35:3843–3857…
  23. Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G Patil, Matei Zaharia, and 1 others. 2025. Llms can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arXiv:2502.07374. find
  24. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988.
  25. Chenxi Liu, Junjie Liang, Yuqi Jia, Bochuan Cao, Yang Bai, Heng Huang, and Xun Chen. 2025. Explore data left behind in reinforcement learning for reasoning language models. arXiv preprint arXiv:2511.04800. — [arXiv:2511.04800](https://arxiv.org/abs/2511.04800) find
  26. Junyu Luo, Xiao Luo, Xiusi Chen, Zhiping Xiao, Wei Ju, and Ming Zhang. 2024a. Semi-supervised fine-tuning for large language models. arXiv preprint arXiv:2410.14745. find
  27. Junyu Luo, Xiao Luo, Kaize Ding, Jingyang Yuan, Zhiping Xiao, and Ming Zhang. 2024b. Ro-bustft: Robust supervised fine-tuning for large language models under noisy response. arXiv preprint arXiv:2412.14922. — [arXiv:2412.14922](https://arxiv.org/abs/2412.14922) find
  28. Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. 2025. Aimo-2 winning solution: Building state-of-the-art mathematical reasoning models with openmathreasoning dataset. arXiv preprint arXiv:2504.16891. find
  29. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. Orca: Progressive learning from complex explanation traces of gpt-4. arXiv preprint arXiv:2306.02707.
  30. Gongrui Nan, Siye Chen, Jing Huang, Mengyu Lu, Dexun Wang, Chunmei Xie, Weiqi Xiong, Xianzhou Zeng, Qixuan Zhou, Yadong Li, and Xingzhong Xu. 2025. NGRPO: Negative-enhanced group relative policy optimization. arXiv preprint arXiv:2509.18851. — [arXiv:2509.18851](https://arxiv.org/abs/2509.18851) find
  31. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744.
  32. Zhuoshi Pan, Yu Li, Honglin Lin, Qizhi Pei, Zinan Tang, Wei Wu, Chenlin Ming, H Vicky Zhao, Conghui He, and Lijun Wu. 2025. Lemma: Learning from errors for mathematical advancement in llms. arXiv preprint arXiv:2503.17439. find
  33. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350. find
  34. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  35. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, and 1 others. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261.
  36. Qwen Team. 2024. Qwen2.5: A party of foundation models. find
  37. Yongqi Tong, Dawei Li, Sizhe Wang, Yujia Wang, Fei Teng, and Jingbo Shang. 2024. Can llms learn from previous mistakes? investigating llms’ errors to boost for reasoning. arXiv preprint arXiv:2403.20046.
  38. David Vilares and Carlos Gómez-Rodríguez. 2019. Head-qa: A healthcare dataset for complex reasoning. arXiv preprint arXiv:1906.04701.
  39. Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, and Timothy Baldwin. 2024a. Learning from failure: Integrating negative examples when fine-tuning large language models as agents. arXiv preprint arXiv:2402.11651. — [arXiv:2402.11651](https://arxiv.org/abs/2402.11651) find
  40. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024b. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–9529…
  41. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
  42. Zhaohui Yang, Yuxiao Ye, Shilei Jiang, Chen Hu, Linjing Li, Shihong Deng, and Daxin Jiang. 2025c. Unearthing gems from stones: Policy optimization with negative sample augmentation for LLM reasoning. arXiv preprint arXiv:2505.14403. find
  43. Erxin Yu, Jing Li, Ming Liao, Qi Zhu, Boyang Xue, Minghui Xu, Baojun Wang, Lanqing Hong, Fei Mi, and Lifeng Shang. 2025a. Self-error-instruct: Generalizing from errors for llms mathematical reasoning. arXiv preprint arXiv:2505.22591.
  44. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, and 16 others. 2025b. Dapo: An open-source llm reinforcement learning…
  45. Yige Yuan, Teng Xiao, Shuchang Tao, Xue Wang, Jinyang Gao, Bolin Ding, and Bingbing Xu. 2025. Incentivizing reasoning from weak supervision. arXiv preprint arXiv:2505.20072. — [arXiv:2505.20072](https://arxiv.org/abs/2505.20072) find
  46. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476–15488.
  47. Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, and Huan Liu. 2025. Is chain-of-thought reasoning of llms a mirage? a data distribution lens. arXiv preprint arXiv:2508.01191.
  48. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. 2025. Group sequence policy optimization. Preprint, arXiv:2507.18071.
  49. Xinyu Zhu, Mengzhou Xia, Zhepei Wei, Wei-Lin Chen, Danqi Chen, and Yu Meng. 2025. The surprising effectiveness of negative reinforcement in LLM reasoning. arXiv preprint arXiv:2506.01347. find
  50. Juncheng Wu, Sheng Liu, Haoqin Tu, Hang Yu, Xiaoke Huang, James Zou, Cihang Xie, and Yuyin Zhou. 2025. Knowledge or reasoning? a close look at how llms think across domains. arXiv preprint arXiv:2506.02126. find
  51. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025a. Qwen3 technical report. Preprint, arXiv:2505.09388.
  52. Mutian Yang, Jiandong Gao, and Ji Wu. 2025b. Decoupling knowledge and reasoning in llms: An exploration using cognitive dual-system theory. arXiv preprint arXiv:2507.18178. — [arXiv:2507.18178](https://arxiv.org/abs/2507.18178) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xueyun-2026-learning-mistakes-negative-reasoning-samples/