Elamur Research⌕
☰
home / papers / 2023 / 基于深度学习的代码生成方法研究进展

基于深度学习的代码生成方法研究进展

Summary

This survey, written in Chinese, reviews deep learning methods that turn natural-language requirements into source code in general-purpose languages such as Python, Java and C++. It treats this text-to-code task as separate from code completion. The authors queried six literature databases, including ACM DL, IEEE Xplore, Google Scholar and CNKI. They then filtered the results by hand and used forward and backward snowballing to arrive at a curated set of studies. The survey sorts these approaches into three families. The first, feature-based generation, splits into supervised encoder–decoder models and pre-trained models. The other two families build on it, either by adding retrieval of external code or knowledge, or by adding post-processing steps such as reranking, test-based filtering or compiler feedback. The paper also catalogues the benchmark corpora and evaluation metrics used in this area, noting a shift from translation-style scores toward test-based ones. Finally, it discusses open problems, including ambiguous user intent, very large output search spaces and limited human evaluation. For the library, it serves as a structured map of code-generation research up to the arrival of ChatGPT and GPT-4.

Key points

  • The corpus covers 54 directly relevant papers collected up to November 2022. Output rose sharply after CodeBERT appeared in 2020, with 22 papers in 2022 alone. Industry reports such as PanGu-Coder and AlphaCode are included even though they were not peer-reviewed.
  • The supervised line of work traced in the survey begins with Ling et al. (2016) on card-game code for HearthStone and Magic the Gathering. It continues through syntax-aware decoders: Yin et al.'s AST-action model, abstract syntax networks (ASDL), TranX with reranking, a grammar-based CNN and TreeGen. Iyer et al. contributed context-aware generation along with the CONCODE dataset, which is now part of CodeXGLUE.
  • The post-processing and retrieval families include CodeT (generated tests), CodeRL (reinforcement learning), Jigsaw, and compiler-feedback approaches.
  • The benchmarks discussed include CONCODE, APPS, MBPP, MultiPL-E, DS-1000, AixBench, Lyra, XLCoST and MCoNaLa. The metrics covered are BLEU, CodeBLEU and test-based measures such as SPoC-style pass rates.
  • On bibliographic details: the paper appeared online in October 2023, and the issue is Journal of Software 2024, 35(2): 604–628, doi 10.13328/j.cnki.jos.006981.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cited by

1 paper of our library cites it (of the 140673 whose bibliographies we read); each has a page here.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.13328/j.cnki.jos.006981

@article{2023,
  title = {{基于深度学习的代码生成方法研究进展}},
  author = {杨泽洲 and 陈思榕 and 高翠芸 and 李振昊 and 李戈 and 吕荣聪},
  journal = {软件学报 (Journal of Software)},
  year = {2023},
  doi = {10.13328/j.cnki.jos.006981},
  url = {https://doi.org/10.13328/j.cnki.jos.006981}
}

References

86 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Reply to Proposal No.3172 (Education No.297) of the Third Session of the 13th National Committee of the Chinese People’s Political Consultative Conference. 2020 (in Chinese). http://www.moe.gov.cn/jyb_xxgk/xxgk_jyta/jyta_jiaocaiju/202012/t20201209_504364.html ↗
  2. Lu S, Guo DY, Ren S, Huang JJ, Svyatkovskiy A, Blanco A, Clement C, Drain D, Jiang DX, Tang DY, Li G, Zhou LD, Shou LJ, Zhou L, Tufano M, Gong M, Zhou M, Duan N, Sundaresan N, Deng SK, Fu SY, Liu SJ. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. arXiv:2102.04… find
  3. Little G, Miller RC. Keyword programming in Java. Automated Software Engineering, 2009, 16(1): 37–71. [doi: 10.1007/s10515-008-0041-9] — [doi:10.1007/s10515-008-0041-9](https://doi.org/10.1007/s10515-008-0041-9) find
  4. Gvero T, Kuncak V. Interactive synthesis using free-form queries. In: Proc. of the 37th IEEE/ACM IEEE Int ’l Conf. on Software Engineering. Florence: IEEE, 2015. 689–692. [doi: 10.1109/ICSE.2015.224] — [doi:10.1109/icse.2015.224](https://doi.org/10.1109/icse.2015.224) find
  5. Hindle A, Barr ET, Gabel M, Su ZD, Devanbu P. On the naturalness of software. Communications of the ACM, 2016, 59(5): 122–131. [doi: 10.1145/2902362] — [doi:10.1145/2902362](https://doi.org/10.1145/2902362) find
  6. Chen ZZ, Yan M, Xia X, Liu ZX, Xu Z, Lei Y. Research progress of code naturalness and its application. Ruan Jian Xue Bao/Journal of Software, 2022, 33(8): 3015−3034 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6355.htm [doi: 10.13328/j.cnki. jos.006355] — [doi:10.13328/j.cnki](https://doi.org/10.13328/j.cnki) ↗
  7. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A. Experimentation in Software Engineering. Berlin: Springer, 2012. [doi: 10.1007/978-3-642-29044-2] — [doi:10.1007/978-3-642-29044-2](https://doi.org/10.1007/978-3-642-29044-2) find
  8. Feng ZY, Guo DY, Tang DY, Duan N, Feng XC, Gong M, Shou LJ, Qin B, Liu T, Jiang DX, Zhou M. CodeBERT: A pre-trained model for programming and natural languages. In: Proc. of the 2020 Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, 202… find our card
  9. Christopoulou F, Lampouras G, Gritta M, Zhang GC, Guo YP, Li ZQ, Zhang Q, Xiao M, Shen B, Li L, Yu H, Yan L, Zhou PY, Wang X, Ma YC, Iacobacci I, Wang YS, Liang GT, Wei JS, Jiang X, Wang QX, Liu Q. PanGu-Coder: Program synthesis with function-level language modeling. arXiv:2207.11280, 2022. find
  10. Li YJ, Choi D, Chung J, Kushman N, Schrittwieser J, Leblond R, Eccles T, Keeling J, Gimeno F, Dal Lago A, Hubert T, Choy P, De Masson d’autume C, Babuschkin I, Chen XY, Huang PS, Welbl J, Gowal S, Cherepanov A, Molloy J, Mankowitz DJ, Sutherland Robson E, Kohli P, De Freitas N, Kavukcuoglu K, Vinyal… find our card
  11. OpenAI. ChatGPT. 2022. https://openai.com/blog/chatgpt/ ↗
  12. OpenAI. GPT-4. 2023. https://openai.com/research/gpt-4 ↗
  13. Hu X, Li G, Liu F, Jin Z. Program generation and code completion techniques based on deep learning: Literature review. Ruan Jian Xue Bao/Journal of Software, 2019, 30(5): 1206−1223 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5717.htm [doi: 10.13328/j.cnki.jos.005717] — [doi:10.13328/j.cnki.jos.005717](https://doi.org/10.13328/j.cnki.jos.005717) ↗
  14. Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. In: Proc. of the 27th Int ’l Conf. on Neural Information Processing Systems. Montreal: MIT Press, 2014. 3104–3112. find
  15. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proc. of the 31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 6000–6010. find
  16. Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional Transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1 (Long and Short Papers). Minneapol… find
  17. Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 2020, 36(4): 1234–1240. [doi: 10.1093/bioinformatics/btz682] find our card
  18. Gururangan S, Marasović A, Swayamdipta S, Lo K, Beltagy I, Downey D, Smith NA. Don’t stop pretraining: Adapt language models to domains and tasks. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020. 8342–8360. [doi:… find
  19. Ling W, Blunsom P, Grefenstette E, Hermann KM, Kočiský T, Wang FM, Senior A. Latent predictor networks for code generation. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers). Berlin: The Association for Computer Linguistics, 2016. 599–609. [d… find
  20. Yin PC, Neubig G. A syntactic neural model for general-purpose code generation. In: Proc. of the 55th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers). Vancouver: Association for Computational Linguistics, 2017. 440–450. [doi: 10.18653/v1/P17-1041] find our card
  21. Rabinovich M, Stern M, Klein D. Abstract syntax networks for code generation and semantic parsing. In: Proc. of the 55th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers). Vancouver: Association for Computational Linguistics, 2017. 1139–1149. [doi: 10.18653/v1/P17… find
  22. Iyer S, Konstas I, Cheung A, Zettlemoyer L. Mapping language to code in programmatic context. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: Association for Computational Linguistics, 2018. 1643 –1652. [doi: 10.18653/v1/D18-1192] find
  23. Iyer S, Cheung A, Zettlemoyer L. Learning programmatic idioms for scalable semantic parsing. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int’l Joint Conf. on Natural Language Processing (EMNLP-IJCNLP). Hong Kong: Association for Computational Linguisti… find
  24. Yin PC, Neubig G. TRANX: A transition-based neural abstract syntax parser for semantic parsing and code generation. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing: System Demonstrations. Brussels: Association for Computational Linguistics, 2018. 7–12. [doi: 10.18653/… find
  25. Yin PC, Neubig G. Reranking for neural semantic parsing. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: Association for Computational Linguistics, 2019. 4553–4559. [doi: 10.18653/v1/P19-1447] find
  26. Sun ZY, Zhu QH, Mou LL, Xiong YF, Li G, Zhang L. A grammar-based structural CNN decoder for code generation. Proc. of the AAAI Conf. on Artificial Intelligence, 2019, 33(1): 7055–7062. [doi: 10.1609/aaai.v33i01.33017055] find
  27. Wei BL, Li G, Xia X, Fu ZY, Jin Z. Code generation as a dual task of code summarization. In: Proc. of the 33rd Int’l Conf. on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019. 6563–6573. find
  28. Ye W, Xie R, Zhang JL, Hu TX, Wang XY, Zhang SK. Leveraging code generation to improve code retrieval and summarization via dual learning. In: Proc. of the 2020 Web Conf. Taipei: ACM, 2020. 2309–2319. [doi: 10.1145/3366423.3380295] — [doi:10.1145/3366423.3380295](https://doi.org/10.1145/3366423.3380295) find
  29. Sun ZY, Zhu QH, Xiong YF, Sun YC, Mou LL, Zhang L. TreeGen: A tree-based transformer architecture for code generation. Proc. of the AAAI Conf. on Artificial Intelligence, 2020, 34(5): 8984–8991. [doi: 10.1609/aaai.v34i05.6430] find
  30. Kanade A, Maniatis P, Balakrishnan G, Shi KS. Learning and evaluating contextual embedding of source code. arXiv: 2001.00059, 2020. find
  31. Guo DY, Ren S, Lu S, Feng ZY, Tang DY, Liu SJ, Zhou L, Duan N, Svyatkovskiy A, Fu SY, Tufano M, Deng SK, Clement C, Drain D, Sundaresan N, Yin J, Jiang DX, Zhou M. GraphCodeBERT: Pre-training code representations with data flow. arXiv:2009.08366, 2021. find our card
  32. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI blog, 2019, 1(8): 9. find
  33. Xu FF, Alon U, Neubig G, Hellendoorn VJ. A systematic evaluation of large language models of code. In: Proc. of the 6th ACM SIGPLAN Int’l Symp. on Machine Programming. San Diego: ACM, 2022. 1–10. [doi: 10.1145/3520312.3534862] find
  34. Nijkamp E, Pang B, Hayashi H, Tu LF, Wang H, Zhou YB, Savarese S, Xiong CM. CodeGen: An open large language model for code with multi-turn program synthesis. arXiv:2203.13474, 2023. find
  35. Zan DG, Chen B, Yang DJ, Lin ZQ, Kim M, Guan B, Wang YJ, Chen WZ, Lou JG. CERT: Continual pre-training on sketches for library-oriented code generation. arXiv:2206.06888, 2022. — [arXiv:2206.06888](https://arxiv.org/abs/2206.06888) find
  36. Black S, Gao L, Wang P, Leahy C, Biderman SR. GPT-Neo: Large scale autoregressive language modeling with mesh-TensorFlow, 2021. find
  37. Fried D, Aghajanyan A, Lin J, Wang SD, Wallace E, Shi F, Zhong RQ, Yih WT, Zettlemoyer L, Lewis M. InCoder: A generative model for code infilling and synthesis. arXiv:2204.05999, 2023. find
  38. Clement C, Drain D, Timcheck J, Svyatkovskiy A, Sundaresan N. PyMT5: Multi-mode translation of natural language and Python code with Transformers. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing (EMNLP). Online: Association for Computational Linguistics, 2020. 9052–90… find
  39. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou YQ, Li W, Liu PJ. Exploring the limits of transfer learning with a unified text-to-text Transformer. The Journal of Machine Learning Research, 2020, 21(1): 5485–5551. find
  40. Ahmad W, Chakraborty S, Ray B, Chang KW. Unified pre-training for program understanding and generation. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2021. 2655–2668.… find our card
  41. Wang Y, Wang WS, Joty S, Hoi SCH. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. Punta Cana: Association for Computational Linguistics, 2021. 8696–8708. [doi:… find
  42. Phan L, Tran H, Le D, Nguyen H, Anibal J, Peltekian A, Ye YF. CoTexT: Multi-task learning with code-text Transformer. arXiv:2105.08645, 2021. find
  43. Guo DY, Lu S, Duan N, Wang YL, Zhou M, Yin J. UniXcoder: Unified cross-modal pre-training for code representation. In: Proc. of the 60th Annual Meeting of the Association for Computational Linguistics (Vol. 1: Long Papers). Dublin: Association for Computational Linguistics, 2022. 7212–7225. [doi: 10… find our card
  44. Chen M, Tworek J, Jun H, et al. Evaluating large language models trained on code. arXiv:2107.03374, 2021. find our card
  45. Copilot. 2023. https://github.com/features/copilot/ ↗
  46. CodeForces. 2023. https://codeforces.com/ ↗
  47. Zeng W, Ren XZ, Su T, et al. PanGu-α: Large-scale autoregressive pretrained chinese language models with auto-parallel computation. arXiv:2104.12369, 2021. find
  48. Huawei Cloud. 2023 (in Chinese). https://www.huaweicloud.com/product/ide.html ↗
  49. aiXcoder. 2023. https://www.aixcoder.com/ ↗
  50. CodeGeeX: A Multilingual code generation model. 2022. https://models.aminer.cn/codegeex/blog/ ↗
  51. Ouyang L, Wu J, Jiang X, Almeida D, Wainwright CL, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A, Schulman J, Hilton J, Kelton F, Miller L, Simens M, Askell A, Welinder P, Christiano P, Leike J, Lowe R. Training language models to follow instructions with human feedback. arXiv:2203.02155, 2022. find
  52. Drain D, Hu CR, Wu C, Breslav M, Sundaresan N. Generating code with the help of retrieved template functions and stack overflow answers. arXiv:2104.05310, 2021. — [arXiv:2104.05310](https://arxiv.org/abs/2104.05310) find
  53. Hayati SA, Olivier R, Avvaru P, Yin PC, Tomasic A, Neubig G. Retrieval-based neural code generation. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: Association for Computational Linguistics, 2018. 925–930. [doi: 10.18653/v1/D18-1111] find
  54. Hashimoto TB, Guu K, Oren Y, Liang P. A retrieve-and-edit framework for predicting structured outputs. In: Proc. of the 32nd Int’l Conf. on Neural Information Processing Systems. Montréal: Curran Associates Inc., 2018. 10073–10083. find
  55. Guo DY, Tang DY, Duan N, Zhou M, Yin J. Coupling retrieval and meta-learning for context-dependent semantic parsing. In: Proc. of the 57th Conf. of the Association for Computational Linguistics. Florence: Association for Computational Linguistics, 2019. 855–866. [doi: 10.18653/v1/P19-1082] find
  56. Xu FF, Jiang ZB, Yin PC, Vasilescu B, Neubig G. Incorporating external knowledge through pre-training for natural language to code generation. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020. 6045–6052. [doi: 10.1… find
  57. Yin PC, Deng BW, Chen E, Vasilescu B, Neubig G. Learning to mine aligned code and natural language pairs from stack overflow. In: Proc. of the 15th Int’l Conf. on Mining Software Repositories. Gothenburg: ACM, 2018. 476–486. [doi: 10.1145/3196398.3196408] find our card
  58. Parvez MR, Ahmad W, Chakraborty S, Ray B, Chang KW. Retrieval augmented code generation and summarization. In: Proc. of the 2021 Findings of the Association for Computational Linguistics. Punta Cana: Association for Computational Linguistics, 2021. 2719–2734. [doi: 10.18653/v1/2021.findings-emnlp.23… find
  59. Zhang TY, Yu T, Hashimoto TB, Lewis M, Yih WT, Fried D, Wang SI. Coder reviewer reranking for code generation. arXiv: 2211.16490, 2022. find
  60. Zhou SY, Alon U, Xu FF, Wang ZR, Jiang ZB, Neubig G. DocPrompting: Generating code by retrieving the docs. arXiv:2207.05987, 2023. — [arXiv:2207.05987](https://arxiv.org/abs/2207.05987) find
  61. Zan DG, Chen B, Lin ZQ, Guan B, Wang YJ, Lou JG. When language model meets private library. In: Proc. of the 2022 Findings of the Association for Computational Linguistics. Abu Dhabi: Association for Computational Linguistics, 2022. 277–288. find
  62. Poesia G, Polozov A, Le V, Tiwari A, Soares G, Meek C, Gulwani S. Synchromesh: Reliable code generation from pre-trained language models. arXiv:2201.11227, 2022. find
  63. Jain N, Vaidyanath S, Iyer A, Natarajan N, Parthasarathy S, Rajamani S, Sharma R. Jigsaw: Large language models meet program synthesis. In: Proc. of the 44th Int ’l Conf. on Software Engineering. Pittsburgh: ACM, 2022. 1219 –1231. [doi: 10.1145/3510003.3510203] — [doi:10.1145/3510003.3510203](https://doi.org/10.1145/3510003.3510203) find
  64. Wang X, Wang YS, Wan Y, Mi F, Li YT, Zhou PY, Liu J, Wu H, Jiang X, Liu Q. Compilable neural code generation with compiler feedback. In: Proc. of the 2022 Findings of the Association for Computational Linguistics. Dublin: Association for Computational Linguistics, 2022. 9–19. [doi: 10.18653/v1/2022.… find
  65. Chen B, Zhang FJ, Nguyen A, Zan DG, Lin ZQ, Lou JG, Chen WZ. CodeT: Code generation with generated tests. arXiv:2207.10397, 2022. find
  66. Le H, Wang Y, Gotmare AD, Savarese S, Hoi SCH. CodeRL: Mastering code generation through pretrained models and deep reinforcement learning. arXiv:2207.01780, 2022. find our card
  67. Liang QY, Sun ZY, Zhu QH, Zhang WJ, Yu L, Xiong YF, Zhang L. Lyra: A benchmark for turducken-style code generation. arXiv: 2108.12144, 2022. — [arXiv:2108.12144](https://arxiv.org/abs/2108.12144) find
  68. Zhu M, Jain A, Suresh K, Ravindran R, Tipirneni S, Reddy CK. XLCoST: A benchmark dataset for cross-lingual code intelligence. arXiv:2206.08474, 2022. find
  69. Wang ZR, Cuenca G, Zhou SY, Xu FF, Neubig G. MCoNaLa: A benchmark for code generation from multiple natural languages. In: Proc. of the 2023 Findings of the Association for Computational Linguistics. Dubrovnik: Association for Computational Linguistics, 2023. 265–273. find
  70. Hendrycks D, Basart S, Kadavath S, Mazeika M, Arora A, Guo E, Burns C, Puranik S, He H, Song D, Steinhardt J. Measuring coding challenge competence with Apps. arXiv:2105.09938, 2021. find our card
  71. Austin J, Odena A, Nye M, Bosma M, Michalewski H, Dohan D, Jiang E, Cai C, Terry M, Le Q, Sutton C. Program synthesis with large language models. arXiv:2108.07732, 2021. find our card
  72. Cassano F, Gouwar J, Nguyen D, Nguyen S, Phipps-Costin L, Pinckney D, Yee MH, Zi YT, Anderson CJ, Feldman MQ, Guha A, Greenberg M, Jangda A. MultiPL-E: A scalable and extensible approach to benchmarking neural code generation. arXiv:2208.08227, 2022. find
  73. Lai YH, Li CX, Wang YM, Zhang TY, Zhong RQ, Zettlemoyer L, Yih SWT, Fried D, Wang SD, Yu T. DS-1000: A natural and reliable benchmark for data science code generation. arXiv:2211.11501, 2022. find
  74. Hao YY, Li G, Liu YQ, Miao XW, Zong H, Jiang SY, Liu Y, Wei H. AixBench: A code generation benchmark dataset. arXiv: 2206. 13179, 2022. find
  75. Papineni K, Roukos S, Ward T, Zhu WJ. BLEU: A method for automatic evaluation of machine translation. In: Proc. of the 40th Annual Meeting of the Association for Computational Linguistics. Philadelphia: ACL, 2002. 311–318. [doi: 10.3115/1073083.1073135] find our card
  76. Ren S, Guo DY, Lu S, Zhou L, Liu SJ, Tang DY, Sundaresan N, Zhou M, Blanco A, Ma S. CodeBLEU: A method for automatic evaluation of code synthesis. arXiv:2009.10297, 2020. — [arXiv:2009.10297](https://arxiv.org/abs/2009.10297) find
  77. Kulal S, Pasupat P, Chandra K, Lee M, Padon O, Aiken A, Liang P. SPoC: Search-based pseudocode to code. In: Proc. of the 33rd Int’l Conf. on Neural Information Processing Systems. Vancouver: Curran Associates Inc., 2019. 11906–11917. find
  78. Niu CG, Li CY, Luo B, Ng V. Deep learning meets software engineering: A survey on pre-trained models of source code. In: Proc. of the 31st Int’l Joint Conf. on Artificial Intelligence. Vienna: IJCAI.org, 2022. 5546–5555. find
  79. Wang CZ, Yang YH, Gao CY, Peng Y, Zhang HY, Lyu MR. No more fine-tuning? An experimental evaluation of prompt tuning in code intelligence. In: Proc. of the 30th ACM Joint European Software Engineering Conf. and Symp. on the Foundations of Software Engineering. Singapore: ACM, 2022. 382–394. [doi: 10… — [doi:10.1145/3540250.3549113](https://doi.org/10.1145/3540250.3549113) find
  80. Wei J, Wang XZ, Schuurmans D, Bosma M, Ichter B, Xia F, Chi E, Le Q, Zhou D. Chain-of-thought prompting elicits reasoning in large language models. arXiv:2201.11903, 2023. find our card
  81. Orlanski G, Gittens A. Reading StackOverflow encourages cheating: Adding question text improves extractive code generation. arXiv:2106.04447, 2021. find
  82. Orlanski G, Yang S, Healy M. Evaluating how fine-tuning on bimodal data effects code generation. arXiv:2211.07842, 2022. — [arXiv:2211.07842](https://arxiv.org/abs/2211.07842) find
  83. Xu FF, Vasilescu B, Neubig G. In-IDE code generation from natural language: Promise and challenges. ACM Trans. on Software Engineering and Methodology, 2022, 31(2): 29. [doi: 10.1145/3487569] — [doi:10.1145/3487569](https://doi.org/10.1145/3487569) find
  84. 12):29 –38, 46. Song C, Jiang Y, Yang Y. Programming contribution analysis for using automatic code generation and completion. Information Technology, 2022, 46(12): 29–38, 46 (in Chinese with English abstract). [doi: 10.13274/j.cnki.hdzj.2022.12.006] — [doi:10.13274/j.cnki.hdzj.2022.12.006](https://doi.org/10.13274/j.cnki.hdzj.2022.12.006) find
  85. Chiang WL, Li Z, Lin Z, Sheng Y, Wu Z, Zhang H, Zheng L, Zhuang S, Zhuang Y, Gonzalez JE, Stoica I, Xing EP. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ ↗
  86. Geng XY, Gudibande A, Liu H, Wallace E, Abbeel P, Levine S, Song D. Koala: A dialogue model for academic research. 2023. https://bair.berkeley.edu/blog/2023/04/03/koala/附中文参考文献: [1] 关于政协十三届全国委员会第三次会议第3172号(教育类297号)提案答复的函. 2020. http://www.moe.gov.cn/jyb_xxgk/xxgk_jyta/jyta_jiaocaiju/202012/t20201209… — [doi:10.13328/j.cnki.jos.006355](https://doi.org/10.13328/j.cnki.jos.006355) ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2023-deep-learning-code/