Elamur Research
☰
home / papers / 2023 / Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2305.01210

Summary

Benchmarks for LLM code generation, such as HumanEval, check whether a generated program is correct by running a small set of hand-written tests. Most tasks have fewer than ten, and they are often simple. As a result, buggy programs can pass and scores look better than they should. The authors present EvalPlus, a framework that adds many more test inputs to an existing benchmark automatically. First, ChatGPT reads the reference solution and some example inputs, then proposes difficult seed inputs and corner cases. Next, type-aware mutation turns these seeds into a much larger set of inputs. Program contracts filter out invalid inputs. Each LLM-written program is then checked against the ground-truth implementation by differential testing. An optional greedy set-cover step makes the test suite much smaller while keeping its bug-finding power. Applied to HumanEval, the method produces HumanEval+, which has 80× more tests. Across 26 models, pass@k falls substantially, and some model rankings change. The study shows that reported code-synthesis results overstate real correctness, and that automated testing is a practical way to build stronger benchmarks.

Key points

  • HumanEval+ adds 80× more tests to the 164 Python problems in HumanEval. A reduced version, HumanEval+-mini, is 47× smaller and catches errors almost as well. The reduction keeps code coverage, mutation kills and LLM-sample kills.
  • The evaluation covered 26 LLMs, including GPT-4 and ChatGPT. On HumanEval+, pass@k dropped by up to 19.3–28.9%, depending on k.
  • Rankings changed. On HumanEval+, the 34B WizardCoder-CodeLlama and Phind-CodeLlama models score higher than ChatGPT. Neither did on the original HumanEval.
  • The new tests also exposed errors in some HumanEval reference solutions. In one case, a date-validation solution wrongly rejects 12-31-1999.
  • The tools, the extended datasets and all LLM-generated code are open-source at github.com/evalplus/evalplus. A public leaderboard is maintained, and the PyPI package was installed over 6k times in its first 5 months. The paper was published at NeurIPS 2023.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Computing & GPUs

Cited by

341 papers of our library cite it (of the 140673 whose bibliographies we read); 20 of them have a page here, 321 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.01210 · arXiv 2305.01210

@misc{liu2023is,
  title = {{Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation}},
  author = {Liu, Jiawei and Xia, Chunqiu Steven and Wang, Yuyao and Zhang, Lingming},
  year = {2023},
  doi = {10.48550/arxiv.2305.01210},
  eprint = {2305.01210},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.01210}
}

References

76 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. T. Ahmed and P. Devanbu. Few-shot training llms for project-specific code-summarization. In 37th IEEE/ACM International Conference on Automated Software Engineering, pages 1–5, 2022. find
  2. L. B. Allal, R. Li, D. Kocetkov, C. Mou, C. Akiki, C. M. Ferrandis, N. Muennighoff, M. Mishra, A. Gu, M. Dey, et al. Santacoder: don’t reach for the stars! arXiv preprint arXiv:2301.03988, 2023. find our card
  3. J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, and C. Sutton. Program synthesis with large language models, 2021. find our card
  4. S. Bang, S. Nam, I. Chun, H. Y. Jhoo, and J. Lee. Smt-based translation validation for machine learning compiler. In Computer Aided Verification: 34th International Conference, CAV 2022, Haifa, Israel, August 7–10, 2022, Proceedings, Part II, pages 386–407. Springer, 2022. find
  5. S. Black, L. Gao, P. Wang, C. Leahy, and S. Biderman. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow, Mar. 2021. If you use this software, please cite it using these metadata. find
  6. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. find
  7. T. A. Budd. Mutation analysis of program test data. Yale University, 1980. find
  8. C. Cadar, D. Dunbar, D. R. Engler, et al. Klee: unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, volume 8, pages 209–224, 2008. find
  9. F. Cassano, J. Gouwar, D. Nguyen, S. Nguyen, L. Phipps-Costin, D. Pinckney, M.-H. Yee, Y. Zi, C. J. Anderson, M. Q. Feldman, et al. Multipl-e: A scalable and polyglot approach to benchmarking neural code generation. IEEE Transactions on Software Engineering, 2023. find
  10. S. K. Cha, M. Woo, and D. Brumley. Program-adaptive mutational fuzzing. In 2015 IEEE Symposium on Security and Privacy, pages 725–741. IEEE, 2015. find
  11. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. find our card
  12. W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, I. Stoica, and E. P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. find
  13. B. Code. Starcoder. https://github.com/bigcode-project/starcoder, 2023. ↗
  14. Y. Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In 32nd International Symposium on Software Testing and Analysis (ISSTA), 2023. find
  15. Y. Deng, C. S. Xia, C. Yang, S. D. Zhang, S. Yang, and L. Zhang. Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt. In 46th International Conference on Software Engineering (ICSE), 2024. find
  16. fauxpilot. Fauxpilot: an open-source alternative to github copilot server. //github.com/fauxpilot/fauxpilot, 2022. find
  17. U. Feige. A threshold of ln n for approximating set cover. Journal of the ACM (JACM), 45(4):634–652, 1998. find
  18. D. Fried, A. Aghajanyan, J. Lin, S. Wang, E. Wallace, F. Shi, R. Zhong, S. Yih, L. Zettlemoyer, and M. Lewis. Incoder: A generative model for code infilling and synthesis. In The Eleventh International Conference on Learning Representations, 2023. find
  19. C. Green. Application of theorem proving to problem solving. In Readings in Artificial Intelligence, pages 202–222. Elsevier, 1981. find
  20. S. Gulwani. Automating string processing in spreadsheets using input-output examples. SIGPLAN Not., 46(1):317–330, jan 2011. find
  21. S. Gulwani, O. Polozov, and R. Singh. Program synthesis. Foundations and Trends® in Programming Languages, 4(1-2):1–119, 2017. https: find
  22. T. Hastie, R. Tibshirani, J. H. Friedman, and J. H. Friedman. The elements of statistical learning: data mining, inference, and prediction, volume 2. Springer, 2009. find
  23. C. Holler, K. Herzig, and A. Zeller. Fuzzing with code fragments. In 21st USENIX Security Symposium (USENIX Security 12), pages 445–458, Bellevue, WA, Aug. 2012. USENIX Association. find
  24. M. Ivanković, G. Petrović, R. Just, and G. Fraser. Code coverage at google. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 955–963, 2019. find
  25. S. Iyer, I. Konstas, A. Cheung, and L. Zettlemoyer. Mapping language to code in programmatic context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1643–1652, 2018. find
  26. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. find
  27. N. Jiang, K. Liu, T. Lutellier, and L. Tan. Impact of code language models on automated program repair. arXiv preprint arXiv:2302.05020, 2023. — [arXiv:2302.05020](https://arxiv.org/abs/2302.05020) find
  28. C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023. find our card
  29. A. Kalyan, A. Mohta, O. Polozov, D. Batra, P. Jain, and S. Gulwani. Neural-guided deductive search for real-time program synthesis from examples. In International Conference on Learning Representations, 2018. find
  30. J. C. King. Symbolic execution and program testing. 19(7):385–394, 1976. find
  31. M.-A. Lachaux, B. Roziere, L. Chanussot, and G. Lample. Unsupervised translation of programming languages. arXiv preprint arXiv:2006.03511, 2020. find our card
  32. K. R. M. Leino. Dafny: An automatic program verifier for functional correctness. In International conference on logic for programming artificial intelligence and reasoning, pages 348–370. Springer, 2010. find
  33. Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, et al. Competition-level code generation with alphacode. Science, 378(6624):1092–1097, 2022. find our card
  34. J. Liu, Y. Wei, S. Yang, Y. Deng, and L. Zhang. Coverage-guided tensor compiler fuzzing with joint ir-pass mutation. Proceedings of the ACM on Programming Languages, 6(OOPSLA1):1–26, Apr. 2022. find
  35. Z. Liu, C. Chen, J. Wang, X. Che, Y. Huang, J. Hu, and Q. Wang. Fill in the blank: Context-aware automated text input generation for mobile gui testing. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1355–1367. IEEE, 2023. find
  36. N. P. Lopes, J. Lee, C.-K. Hur, Z. Liu, and J. Regehr. Alive2: bounded translation validation for llvm. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pages 65–79, 2021. find
  37. S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang, et al. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664, 2021. find
  38. Z. Luo, C. Xu, P. Zhao, Q. Sun, X. Geng, W. Hu, C. Tao, J. Ma, Q. Lin, and D. Jiang. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568, 2023. find
  39. Z. Manna and R. J. Waldinger. Toward automatic program synthesis. Communications of the ACM, 14(3):151–165, 1971. find
  40. W. M. McKeeman. Differential testing for software. Digital Technical Journal, 10(1):100–107, 1998. find
  41. B. Meyer. Applying’design by contract’. Computer, 25(10):40–51, 1992. find
  42. Microsoft. GitHub Copilot – Your AI pair programmer. https://github.com/features/copilot, 2023. Communications of the ACM, ↗
  43. B. P. Miller, L. Fredriksen, and B. So. An empirical study of the reliability of unix utilities. Communications of the ACM, 33(12):32–44, 1990. find
  44. G. C. Necula. Translation validation for an optimizing compiler. In Proceedings of the ACM SIGPLAN 2000 conference on Programming language design and implementation, pages 83–94, 2000. find
  45. E. Nijkamp, H. Hayashi, C. Xiong, S. Savarese, and Y. Zhou. Codegen2: Lessons for training llms on programming and natural languages. arXiv preprint, 2023. find
  46. E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong. Codegen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations, 2023. find
  47. P. Oehlert. Violating assumptions with fuzzing. IEEE Security & Privacy, 3(2):58–62, 2005. find
  48. OpenAI. Chatgpt: Optimizing language models for dialogue. https://openai.com/blog/chatgpt/, 2022. ↗
  49. OpenAI. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023. find our card
  50. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002. find our card
  51. G. Petrović and M. Ivanković. State of mutation testing at google. In Proceedings of the 40th international conference on software engineering: Software engineering in practice, pages 163–171, 2018. find
  52. Phind. Phind/phind-codellama-34b-v2 · hugging face. https://huggingface.co/Phind/Phind-CodeLlama-34B-v2, 2023. ↗
  53. G. Rothermel, M. J. Harrold, J. Von Ronne, and C. Hong. Empirical studies of test-suite reduction. Software Testing, Verification and Reliability, 12(4):219–249, 2002. find
  54. B. Rozière, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023. find our card
  55. B. Roziere, J. M. Zhang, F. Charton, M. Harman, G. Synnaeve, and G. Lample. Leveraging automated unit tests for unsupervised code translation. arXiv preprint arXiv:2110.06773, 2021. find
  56. M. Security. jsfunfuzz. https://github.com/MozillaSecurity/funfuzz, 2007. ↗
  57. K. Serebryany. Continuous fuzzing with libfuzzer and addresssanitizer. In 2016 IEEE Cybersecurity Development (SecDev), pages 157–157. IEEE, 2016. find
  58. D. E. Shaw, W. R. Swartout, and C. C. Green. Inferring lisp programs from examples. In IJCAI, volume 75, pages 260–267, 1975. find
  59. A. Shi, A. Gyori, M. Gligoric, A. Zaytsev, and D. Marinov. Balancing trade-offs in test-suite reduction. In Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering, pages 246–256, 2014. find
  60. Stability-AI. Stablelm: Stability ai language models. https://github.com/Stability-AI/StableLM, 2023. ↗
  61. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. find
  62. R. J. Waldinger and R. C. Lee. Prow: A step toward automatic program writing. In Proceedings of the 1st international joint conference on Artificial intelligence, pages 241–252, 1969. find
  63. B. Wang and A. Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/mesh-transformer-jax, May 2021. ↗
  64. Y. Wang, H. Le, A. D. Gotmare, N. D. Bui, J. Li, and S. C. Hoi. Codet5+: Open code large language models for code understanding and generation. arXiv preprint arXiv:2305.07922, 2023. find
  65. Y. Wei, C. S. Xia, and L. Zhang. Copiloting the copilots: Fusing large language models with completion engines for automated program repair. arXiv preprint arXiv:2309.00608, 2023. — [arXiv:2309.00608](https://arxiv.org/abs/2309.00608) find
  66. D. Winterer, C. Zhang, and Z. Su. On the unusual effectiveness of type-aware operator mutations for testing smt solvers. Proceedings of the ACM on Programming Languages, 4(OOPSLA):1–25, 2020. find
  67. C. S. Xia, M. Paltenghi, J. L. Tian, M. Pradel, and L. Zhang. Universal fuzzing via large language models. In 46th International Conference on Software Engineering (ICSE), 2024. find
  68. C. S. Xia, Y. Wei, and L. Zhang. Automated program repair in the era of large pre-trained language models. In Proceedings of the 45th International Conference on Software Engineering (ICSE 2023). Association for Computing Machinery, 2023. find
  69. C. S. Xia and L. Zhang. Less training, more repairing please: revisiting automated program repair via zero-shot learning. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 959–971, 2022. find
  70. F. F. Xu, U. Alon, G. Neubig, and V. J. Hellendoorn. A systematic evaluation of large language models of code. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, pages 1–10, 2022. find
  71. C. Yang, Y. Deng, R. Lu, J. Yao, J. Liu, R. Jabbarvand, and L. Zhang. White-box compiler fuzzing empowered by large language models, 2023. find
  72. X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in c compilers. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI ’11, page 283–294, New York, NY, USA, 2011. Association for Computing Machinery. find
  73. T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li, Q. Yao, S. Roman, et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 201… find our card
  74. M. Zalewski. American fuzzing lop (afl). https://lcamtuf.coredump.cx/afl/, 2018. ↗
  75. L. Zhang, D. Marinov, L. Zhang, and S. Khurshid. An empirical study of junit test-suite reduction. In 2011 IEEE 22nd International Symposium on Software Reliability Engineering, pages 170–179. IEEE, 2011. find
  76. Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, Z. Wang, L. Shen, A. Wang, Y. Li, et al. Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x. arXiv preprint arXiv:2303.17568, 2023. Table 5: Overview of evaluated models. Model Name CodeGen [46] I N C ODE… find our card