Elamur Research⌕
☰
home / papers / 2024 / Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step

Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step

Paper · 2024 · Li Zhong, Zilong Wang and Jingbo Shang

Summary

Code produced by large language models often fails tests. Earlier repair methods give the model only end-of-run feedback, such as pass/fail results or error messages, and treat the program as one opaque unit. This paper introduces LDB (Large Language Model Debugger), which mimics how programmers set breakpoints and inspect state.

For a failing visible test, LDB works in three stages:

  • Profiling: it builds the program's control flow graph, records the path of basic blocks taken during execution, and captures variable values around each block.
  • Debugging: an LLM checks each block against the task description, labelling it correct or incorrect and explaining why.
  • Regeneration: the model rewrites the program based on these verdicts.

The loop repeats until the visible tests pass or an iteration limit is reached. Very long traces are subsampled to fit the context window.

Evaluated on HumanEval, MBPP and TransCoder with several backbones, LDB raises accuracy by up to 9.8%. It also uncovers bugs in programs produced by stronger generators. The work shows that intermediate runtime state is a strong signal for LLM-based program repair.

Key points

  • Benchmarks are HumanEval and MBPP (text-to-code) and TransCoder (code translation). Hidden tests are reserved for final evaluation only.
  • Backbone LLMs include GPT-3.5, StarCoder and CodeLlama. LDB improves on baselines by up to 9.8% and further refines seed programs from GPT-4 and Reflexion.
  • "Selective debugging" samples N_b blocks from long traces caused by loops or recursion, keeping prompts within token limits. "Batch debugging" is added to improve efficiency.
  • A key limitation is that LDB needs correct visible test cases. Debugging without test cases is left as an open question.
  • Code is available at github.com/FloridSleeves/LLMDebugger.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California San DiegoSan Diego · United States89 works · 54,032 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksInformation Systems · 1,730 worksSoftware · 119 worksComputer Science Applications · 131 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Li Zhong, Zilong Wang and Jingbo Shang. 2024. Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step. arXiv:2402.16906.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2402.16906 · arXiv 2402.16906

@misc{zhong2024debug,
  title = {{Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step}},
  author = {Li Zhong, Zilong Wang and Jingbo Shang},
  year = {2024},
  doi = {10.48550/arxiv.2402.16906},
  eprint = {2402.16906},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.16906}
}

References

73 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Alfred V Aho and Ravi Sethi Jeffrey D Ullman. 2015. ,“compilers-principles, techniques, and tools”, pearson education asia, 2007. find
  3. Frances E Allen. 1970. Control flow analysis. ACM Sigplan Notices, 5(7):1–19. find
  4. Glenn Ammons and James R Larus. 1998. Improving data-flow analysis with path profiles. In Proceedings of the ACM SIGPLAN 1998 conference on Programming language design and implementation, pages 72–84. find
  5. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732.
  6. Thomas Ball and James R Larus. 1994. Optimally profiling and tracing programs. ACM Transactions on Programming Languages and Systems (TOPLAS), 16(4):1319–1360. find
  7. Thomas Ball and James R Larus. 1996. Efficient path profiling. In Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29, pages 46–57. IEEE. find
  8. Angelica Chen, Jérémy Scheurer, Tomasz Korbak, Jon Ander Campos, Jun Shern Chan, Samuel R Bowman, Kyunghyun Cho, and Ethan Perez. 2023a. Improving code generation by training with natural language feedback. arXiv preprint arXiv:2303.16749. find
  9. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2022. Codet: Code generation with generated tests. In The Eleventh International Conference on Learning Representations. find
  10. Hongqiao Chen, Kexun Zhang, Lei Li, and William Yang Wang. 2023b. Tooldec: Syntax error-free and generalizable tool use for llms via finite-state decoding. In The 3rd Workshop on Mathematical Reasoning and AI at NeurIPS’23. find
  11. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  12. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023c. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128. find
  13. Xinyun Chen, Chang Liu, and Dawn Song. 2018. Execution-guided neural program synthesis. In International Conference on Learning Representations. find
  14. Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, et al. 2022. Binding language models in symbolic languages. In The Eleventh International Conference on Learning Representations.
  15. John Cocke. 1970. Global common subexpression elimination. In Proceedings of a symposium on Compiler optimization, pages 20–24. find
  16. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2023. Chain-of-verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495.
  17. Zhiyu Fan, Xiang Gao, Martin Mirchev, Abhik Roychoudhury, and Shin Hwei Tan. 2023. Automated repair of programs from large language models. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1469–1481. IEEE. find
  18. Python Software Foundation. 2001. pdb — the python debugger. https://docs.python.org/3/library/pdb.html. ↗
  19. Tao Ge, Jing Hu, Xun Wang, Si-Qing Chen, and Furu Wei. 2023. In-context autoencoder for context compression in a large language model. arXiv preprint arXiv:2307.06945.
  20. Qiuhan Gu. 2023. Llm-based code generation method for golang compiler testing. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 2201–2203. find
  21. Priyanshu Gupta, Avishree Khare, Yasharth Bajpai, Saikat Chakraborty, Sumit Gulwani, Aditya Kanade, Arjun Radhakrishna, Gustavo Soares, and Ashish Tiwari. 2023. Grace: Language models meet code edits. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the… find
  22. Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. 2023. find
  23. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352. — MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
  24. Mengkang Hu, Yao Mu, Xinmiao Yu, Mingyu Ding, Shiguang Wu, Wenqi Shao, Qiguang Chen, Bin Wang, Yu Qiao, and Ping Luo. 2023. Tree-planner: Efficient close-loop task planning with large language models. arXiv preprint arXiv:2310.08582. — [arXiv:2310.08582](https://arxiv.org/abs/2310.08582) find
  25. Xueyu Hu, Kun Kuang, Jiankai Sun, Hongxia Yang, and Fei Wu. 2024. Leveraging print debugging to improve code generation in large language models. arXiv preprint arXiv:2401.05319. — [arXiv:2401.05319](https://arxiv.org/abs/2401.05319) find
  26. Baizhou Huang, Shuai Lu, Weizhu Chen, Xiaojun Wan, and Nan Duan. 2023a. Enhancing large language models in coding through multi-perspective self-consistency. arXiv preprint arXiv:2309.17272. find
  27. Haochen Huang, Bingyu Shen, Li Zhong, and Yuanyuan Zhou. 2023b. Protecting data integrity of web applications with database constraints inferred from application code. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Vo… find
  28. Haochen Huang, Chengcheng Xiang, Li Zhong, and Yuanyuan Zhou. 2021. {PYLIVE}:{On-the-Fly} code change for python-based online services. In 2021 USENIX Annual Technical Conference (USENIX ATC 21), pages 349–363. find
  29. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023c. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798.
  30. Kai Huang, Xiangxin Meng, Jian Zhang, Yang Liu, Wenjie Wang, Shuhao Li, and Yuqing Zhang. 2023d. An empirical study on fine-tuning large language models of code for automated program repair. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 1162– 1174. IEE… find
  31. Shuyang Jiang, Yuhao Wang, and Yu Wang. 2023. Self-evolve: A code evolution framework via large language models. arXiv preprint arXiv:2306.02907. — [arXiv:2306.02907](https://arxiv.org/abs/2306.02907) find
  32. Matthew Jin, Syed Shahriar, Michele Tufano, Xin Shi, Shuai Lu, Neel Sundaresan, and Alexey Svyatkovskiy. 2023. Inferfix: End-to-end program repair with llms. arXiv preprint arXiv:2303.07263. — [arXiv:2303.07263](https://arxiv.org/abs/2303.07263) find
  33. James R Larus. 1999. Whole program paths. ACM SIGPLAN Notices, 34(5):259–269. find
  34. Hung Le, Hailin Chen, Amrita Saha, Akash Gokul, Doyen Sahoo, and Shafiq Joty. 2023. Codechain: Towards modular code generation through chain of self-revisions with representative sub-modules. arXiv preprint arXiv:2310.08992. — [arXiv:2310.08992](https://arxiv.org/abs/2310.08992) find
  35. Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. 2022. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 35:21314–21328.
  36. Jian Li, Yue Wang, Michael R. Lyu, and Irwin King. 2018. Code completion with neural attention and pointer networks. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI’18, page 4159–25. AAAI Press. find
  37. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161. find
  38. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode. Science, 378(6624):1092–1097.
  39. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050.
  40. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. find
  41. Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. 2023. Octopack: Instruction tuning code large language models. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following. find
  42. Ansong Ni, Jeevana Priya Inala, Chenglong Wang, Alex Polozov, Christopher Meek, Dragomir Radev, and Jianfeng Gao. 2022. Learning math reasoning from self-sampled correct and partially-correct solutions. In The Eleventh International Conference on Learning Representations. find
  43. Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-tau Yih, Sida Wang, and Xi Victoria Lin. 2023. Lever: Learning to verify language-to-code generation with execution. In International Conference on Machine Learning, pages 26106–26128. PMLR.
  44. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations. find
  45. Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2023. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334.
  46. Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2023. Examining zero-shot vulnerability repair with large language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2339–2356. IEEE. find
  47. Reese T Prosser. 1959. Applications of boolean matrices to the analysis of flow diagrams. In Papers presented at the December 1-3, 1959, eastern joint IRE-AIEE-ACM computer conference, pages 133– 138. find
  48. Zeeshan Rasheed, Muhammad Waseem, Mika Saari, Kari Systä, and Pekka Abrahamsson. 2024. Code-pori: Large scale model for autonomous software development by using multi-agents. arXiv preprint arXiv:2402.01411. — [arXiv:2402.01411](https://arxiv.org/abs/2402.01411) find
  49. Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In Proceedings of the 35th ACM SIGPLAN conference on programming language design and implementation, pages 419–428. find
  50. Eric Rotenberg, Steve Bennett, and James E Smith. 1996. Trace cache: a low latency approach to high bandwidth instruction fetching. In Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29, pages 24–34. IEEE. find
  51. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950.
  52. Baptiste Roziere, Marie-Anne Lachaux, Lowik Chanussot, and Guillaume Lample. 2020. Unsupervised translation of programming languages. Advances in Neural Information Processing Systems, 33:20601– 20611.
  53. Timothy Sherwood, Erez Perelman, and Brad Calder. 2001. Basic block distribution analysis to find periodic behavior and simulation points in applications. In Proceedings 2001 International Conference on Parallel Architectures and Compilation Techniques, pages 3–14. IEEE. find
  54. Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I Wang. 2022. Natural language to code translation with execution. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3533–3546.
  55. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems.
  56. Richard Stallman, Roland Pesch, Stan Shebs, et al. 1988. Debugging with gdb. Free Software Foundation, 675. find
  57. Zilu Tang, Mayank Agarwal, Alexander Shypula, Bailin Wang, Derry Wijaya, Jie Chen, and Yoon Kim. 2023. Explain-then-translate: an analysis on improving program translation with self-generated explanations. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1741–1788. find
  58. Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, et al. 2024. Chain-of-table: Evolving tables in the reasoning chain for table understanding. arXiv preprint arXiv:2401.04398.
  59. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  60. Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. 2023. Fine-grained human feedback gives better rewards for language model training. arXiv preprint arXiv:2306.01693.
  61. Pengcheng Yin and Graham Neubig. 2017. A syntactic neural model for general-purpose code generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 440–450.
  62. Zhiqiang Yuan, Junwei Liu, Qiancheng Zi, Ming-wei Liu, Xin Peng, and Yiling Lou. 2023. Evaluating instruction-tuned large language models on code comprehension and generation. arXiv preprint arXiv:2308.01240. — [arXiv:2308.01240](https://arxiv.org/abs/2308.01240) find
  63. Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, and Gust Verbruggen. 2022a. Repairing bugs in python assignments using large language models. arXiv preprint arXiv:2209.14876. — [arXiv:2209.14876](https://arxiv.org/abs/2209.14876) find
  64. Jialu Zhang, De Li, John Charles Kolesar, Hanyuan Shi, and Ruzica Piskac. 2022b. Automated feedback generation for competition-level code. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, pages 1–13. find
  65. Jialu Zhang, Todd Mytkowicz, Mike Kaufman, Ruzica Piskac, and Shuvendu K Lahiri. 2022c. Using pre-trained language models to resolve textual and semantic merge conflicts (experience paper). In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 77–88. find
  66. Kexun Zhang, Danqing Wang, Jingtao Xia, William Yang Wang, and Lei Li. 2023a. Algo: Synthesizing algorithmic programs with generated oracle verifiers. arXiv preprint arXiv:2305.14591.
  67. Tianyi Zhang, Tao Yu, Tatsunori Hashimoto, Mike Lewis, Wen-tau Yih, Daniel Fried, and Sida Wang. 2023b. Coder reviewer reranking for code generation. In International Conference on Machine Learning, pages 41832–41846. PMLR. find
  68. Li Zhong. 2023. A survey of prevent and detect access control vulnerabilities. arXiv preprint arXiv:2304.10600. — [arXiv:2304.10600](https://arxiv.org/abs/2304.10600) find
  69. Li Zhong and Zilong Wang. 2023. A study on robustness and reliability of large language model code generation. arXiv preprint arXiv:2308.10335. — [arXiv:2308.10335](https://arxiv.org/abs/2308.10335) find
  70. Li Zhong, Chengcheng Xiang, Haochen Huang, Bingyu Shen, Eric Mugnier, and Yuanyuan Zhou. 2024. Effective bug detection with unused definitions. In Proceedings of the Nineteenth European Conference on Computer Systems, pages 720–735. find
  71. Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. 2023a. Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406. find
  72. Aojun Zhou, Ke Wang, Zimu Lu, Weikang Shi, Sichun Luo, Zipeng Qin, Shaoqing Lu, Anya Jia, Linqi Song, Mingjie Zhan, et al. 2023b. Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification. arXiv preprint arXiv:2308.07921.
  73. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, et al. 2022. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning Representations.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhong-2024-debug-like-human-large-language/