Elamur Research⌕
☰
home / papers / 2026 / Mistake Notebook Learning: Batch-Clustered Failures for Training-Free Agent Adaptation

Mistake Notebook Learning: Batch-Clustered Failures for Training-Free Agent Adaptation

Paper · 2026 · Xuanbo Su, Yingfang Zhang, Hao Luo et al.

Summary

LLM agents that run for long periods keep failing on tasks but rarely turn those failures into lasting lessons, so the same errors recur. Fine-tuning is costly and fragile. Existing training-free options either optimize a single prompt or store individual past episodes without generalizing from them. The authors propose Mistake Notebook Learning (MNL). A frozen Tuning Model answers a batch of tasks. A separate Tuner Model then groups the batch by subject, analyzes the failures, and writes structured "mistake notes" into an external memory, which is later retrieved by embedding similarity. A revised memory is kept only if it raises accuracy on that batch; otherwise the previous state is restored. Correctness is judged by ground truth for math and Text-to-SQL, or by an LLM judge for agentic tasks such as Mind2Web and AppWorld. The memory can also steer test-time scaling. On Qwen3-8B, MNL beats the vanilla model and LoRA while keeping memory small, and on GSM8K it comes close to full SFT. This gives cheap, gradient-free adaptation, and the code is public.

Key points

  • Benchmarks: AIME, GSM8K, KaggleDBQA, Spider (Supervised Evolution), plus Mind2Web and AppWorld (Self-Evolution with an LLM judge).
  • Qwen3-8B results:
    • GSM8K: MNL 93.9% vs. base 91.8%, LoRA 92.2%, and full SFT 94.3%.
    • Spider: MNL 71.7% vs. vanilla 68.9%, LoRA 69.8%, and SFT 79.0%.
  • Batch clustering matters on KaggleDBQA. Batch size 16 gave 28% accuracy with 23 memory entries, while batch size 1 gave 24% with 69 entries.
  • A single training epoch works best (28.1% test accuracy on KaggleDBQA). At a second epoch, training accuracy rose to 62.1% but test accuracy fell to 23.2%, a sign of overfitting.
  • Using DeepSeek-V3.2-Exp as the tuner beat self-tuning on KaggleDBQA (31.0% vs. 28.0%), and both reached 30.0% on AIME 2025.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harbin Institute of TechnologyHarbin · China158 works · 28,841 citations🎓Jilin UniversityChangchun · China25 works · 5,252 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Xuanbo Su, Yingfang Zhang, Hao Luo, Xiaoteng Liu and Leo Huang. 2026. Mistake Notebook Learning: Batch-Clustered Failures for Training-Free Agent Adaptation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.719

@inproceedings{su2026mistake,
  title = {{Mistake Notebook Learning: Batch-Clustered Failures for Training-Free Agent Adaptation}},
  author = {Xuanbo Su, Yingfang Zhang, Hao Luo, Xiaoteng Liu and Leo Huang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.719},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.719}
}

References

40 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. 2024. Learning from mistakes makes llm better reasoner. Preprint, arXiv:2310.20689. find
  2. Yuzheng Cai, Siqi Cai, Yuchen Shi, Zihan Xu, Lichao Chen, Yulei Qin, Xiaoyu Tan, Gang Li, Zongyi Li, Haojia Lin, Yong Mao, Ke Li, and Xing Sun. 2025. Training-free group relative policy optimization. Preprint, arXiv:2510.08191. — [arXiv:2510.08191](https://arxiv.org/abs/2510.08191) find
  3. Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. 2023. Fireact: Toward language agent fine-tuning. Preprint, arXiv:2310.05915.
  4. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  5. DeepSeek-AI. 2025a. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  6. DeepSeek-AI. 2025b. Deepseek-v3.2-exp: Boosting long-context efficiency with deepseek sparse attention. find
  7. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2web: Towards a generalist agent for the web. Preprint, arXiv:2306.06070.
  8. Yufeng Du, Minyang Tian, Srikanth Ronanki, Subendhu Rongali, Sravan Babu Bodapati, Aram Galstyan, Azton Wells, Roy Schwartz, Eliu A. Huerta, and Hao Peng. 2025. Context length alone hurts llm performance despite perfect retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2… find
  9. Jiawei Gu, Xinyi Jiang, Zichu Shi, Haoyu Tan, Xing Zhai, Can Xu, Wei Li, Yan Shen, Shuo Ma, Heng Liu, and 1 others. 2025. A survey on llm-as-a-judge. Preprint, arXiv:2411.15594. find
  10. Yujin Kim, Euiin Yi, Minu Kim, Se-Young Yun, and Taehyeon Kim. 2025. Guiding reasoning in small language models with llm assistance. Preprint, arXiv:2504.09923.
  11. Chia-Hsuan Lee, Oleksandr Polozov, and Matthew Richardson. 2021. KaggleDBQA: Realistic evaluation of text-to-SQL parsers. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: L… find
  12. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173.
  13. Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. 2024. Appworld: A controllable world of apps and people for benchmarking interactive coding agents. Preprint, arXiv:2407.18901.
  14. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with… find
  15. Mathematical Association of America. 2025. American Invitational Mathematics Examination (AIME). https://artofproblemsolving.com/wiki/ index.php/AIME_Problems_and_Solutions. Accessed: 2025-12-31. ↗
  16. Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. 2025. Reasoningbank: Scaling agent self-evolving with reasoning memory.… — [arXiv:2509.25140](https://arxiv.org/abs/2509.25140) find
  17. Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. Preprint, arXiv:2304.03442. find
  18. Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. Automatic prompt optimization with "gradient descent" and beam search. Preprint, arXiv:2305.03495.
  19. Xin Qiu, Yulu Gan, Conor F. Hayes, Qiyao Liang, Elliot Meyerson, Babak Hodjat, and Risto Miikkulainen. 2025. Evolution strategies at scale: Llm fine-tuning beyond reinforcement learning. Preprint, arXiv:2509.24372. — [arXiv:2509.24372](https://arxiv.org/abs/2509.24372) find
  20. Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Preprint, arXiv:2303.11366.
  21. Zeyi Sun, Ziyu Liu, Yuhang Zang, Yuhang Cao, Xiaoyi Dong, Tong Wu, Dahua Lin, and Jiaqi Wang. 2025. Seagent: Self-evolving computer use agent with autonomous learning from experience. Preprint, arXiv:2508.04700. — [arXiv:2508.04700](https://arxiv.org/abs/2508.04700) find
  22. Yongqi Tong, Dawei Li, Sizhe Wang, Yujia Wang, Fei Teng, and Jingbo Shang. 2024. Can LLMs learn from previous mistakes? investigating LLMs’ errors Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jir…
  23. Yu Wang, Ryuichi Takanobu, Zhiqi Liang, Yuzhen Mao, Yuanzhe Hu, Julian McAuley, and Xiaojian Wu. 2025a. Mem-α: Learning memory construction via reinforcement learning. ArXiv, abs/2509.25911. — [arXiv:2509.25911](https://arxiv.org/abs/2509.25911) find
  24. Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. 2025b. Agent workflow memory. In Forty-second International Conference on Machine Learning.
  25. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. Preprint, arXiv:2201.11903.
  26. LLM-Core Xiaomi. 2026. Mimo-v2-flash technical report. Preprint, arXiv:2601.02780.
  27. Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Kristian Kersting, Jeff Z. Pan, Hinrich Schütze, Volker Tresp, and Yunpu Ma. 2025. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning. Preprint, arXiv:2508…
  28. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  29. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2024. Large language models as optimizers. Preprint, arXiv:2309.03409.
  30. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. Preprint, arXiv:2210.03629.
  31. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, and 16 others. 2025. Dapo: An open-source llm reinforcement learning s…
  32. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2019. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. Preprint, arXiv:1809.08887.
  33. Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. 2023. Agenttuning: Enabling generalized agent abilities for llms. Preprint, arXiv:2310.12823.
  34. Yunpeng Zhai, Shuchang Tao, Cheng Chen, Anni Zou, Ziqian Chen, Qingxu Fu, Shinji Mai, Li Yu, Jiaji Deng, Zouying Cao, Zhaoyang Liu, Bolin Ding, and Jingren Zhou. 2025. Agentevolver: Towards efficient self-evolving agent system. Preprint, arXiv:2511.10395. — [arXiv:2511.10395](https://arxiv.org/abs/2511.10395) find
  35. Qizheng Zhang, Changran Hu, Shubhangi Upasani, Boyuan Ma, Fenglu Hong, Vamsidhar Kamanuru, Jay Rainton, Chen Wu, Mengmeng Ji, Hanchen Li, Urmish Thakker, James Zou, and Kunle Olukotun. 2025. Agentic context engineering: Evolving contexts for self-improving language models. Preprint, arXiv:2510.04618… find
  36. Tianjun Zhang, Aman Madaan, Luyu Gao, Steven Zheng, Swaroop Mishra, Yiming Yang, Niket Tandon, and Uri Alon. 2024. In-context principle learning from mistakes. Preprint, arXiv:2402.05403.
  37. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. Expel: Llm agents are experiential learners. Preprint, arXiv:2308.10144. find
  38. Longtao Zheng, Rundong Wang, Xinrun Wang, and Bo An. 2024. Synapse: Trajectory-as-exemplar prompting with memory for computer control. In International Conference on Representation Learning, volume 2024, pages 19036–19066. find
  39. Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, and Jun Wang. 2025. Memento: Fine-tuning llm agents without fine-tuning llms. Preprint, arXiv:2508.16153. — [arXiv:2508.16153](https://arxiv.org/abs/2508.16153) find
  40. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2023. Large language models are human-level prompt engineers. Preprint, arXiv:2211.01910.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/su-2026-mistake-notebook-learning-batch-clustered/