Elamur Research⌕
☰
home / papers / 2017 / A Survey of Machine Learning for Big Code and Naturalness

A Survey of Machine Learning for Big Code and Naturalness

Paper · 2017 · Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu et al.

Summary

This survey maps out a young field that applies machine learning to source code. It is framed by the "naturalness hypothesis": code is written by people for people, so large code corpora show statistical regularities similar to those in natural language, and tools can exploit them. The authors (Microsoft Research, UCL, UC Davis, Edinburgh/Turing Institute) first compare programming and natural languages and explain how the differences shape model design. They then organise probabilistic models of code into a taxonomy based on the design principles behind each model. The models covered include n-gram language models, deep learning approaches, and representations built on sequences, trees and continuous embeddings, as opposed to simple bag-of-words views. Next, they survey where these models are used, including tools that make recommendations to developers, finding and fixing bugs, analysing programs, and automatically generating programs, and then discuss open challenges. The main argument is that software engineering and programming languages research should complement purely formal, worst-case reasoning with data-driven models tuned to the common case. NLP went through a similar shift away from rule-based systems.

Key points

  • Hindle et al. provided the first empirical support for naturalness, showing that models built for natural language worked surprisingly well on code.
  • "Big code" refers to the open-source data now available, which amounts to billions of code tokens plus millions of commits, bug fixes and reviews.
  • Table 4 lists 12 public datasets. Examples include a corpus of 150k Python functions with docstrings (Barone and Sennrich), 800k+ repositories (Dyer et al.), JavaScript/Python AST collections (Raychev et al.), and Python code paired with English/Japanese pseudocode (Oda et al.).
  • The authors maintain living resources at learnbigcode.github.io and ml4code.github.io, which accept community contributions.
  • Related fields include mining software repositories (MSR) and probabilistic programming. The authors describe probabilistic programming as the "dual" direction, since it uses PL concepts to help write ML, while this field uses ML to help write programs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom381 works · 80,747 citations🎓University College LondonLondon · United Kingdom107 works · 22,923 citations🔬The Alan Turing InstituteLondon · United Kingdom31 works · 4,535 citations🎓University of California, DavisDavis · United States34 works · 42,940 citations🏢Microsoft Research (United Kingdom)Cambridge · United Kingdom11 works · 3,215 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

Cite

Cite as Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu and Charles Sutton. 2017. A Survey of Machine Learning for Big Code and Naturalness. arXiv:1709.06182.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1709.06182 · arXiv 1709.06182

@misc{allamanis2017survey,
  title = {{A Survey of Machine Learning for Big Code and Naturalness}},
  author = {Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu and Charles Sutton},
  year = {2017},
  doi = {10.48550/arxiv.1709.06182},
  eprint = {1709.06182},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1709.06182}
}

References

201 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mithun Acharya, Tao Xie, Jian Pei, and Jun Xu. 2007. Mining API patterns as partial orders from source code: from usage scenarios to specifications. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/… find
  2. Karan Aggarwal, Mohammad Salameh, and Abram Hindle. 2015. Using machine translation for converting Python 2 to Python 3 code. Technical Report. find
  3. Alex A Alemi, Francois Chollet, Geoffrey Irving, Christian Szegedy, and Josef Urban. 2016. DeepMath–Deep Sequence Models for Premise Selection. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS).
  4. Miltiadis Allamanis, Earl T. Barr, Christian Bird, Premkumar Devanbu, Mark Marron, and Charles Sutton. 2016. Mining Semantic Loop Idioms from Big Code. Technical Report. https://www.microsoft.com/en-us/research/publication/mining-semantic-loop-idioms-big-code/ ↗
  5. Miltiadis Allamanis, Earl T Barr, Christian Bird, and Charles Sutton. 2014. Learning Natural Coding Conventions. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  6. Miltiadis Allamanis, Earl T Barr, Christian Bird, and Charles Sutton. 2015. Suggesting accurate method and class names. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FSE). find
  7. Miltiadis Allamanis and Marc Brockschmidt. 2017. SmartPaste: Learning to Adapt Source Code. arXiv preprint arXiv:1705.07867 (2017). — [arXiv:1705.07867](https://arxiv.org/abs/1705.07867) find
  8. Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. 2018. Learning to Represent Programs with Graphs. In Proceedings of the International Conference on Learning Representations (ICLR).
  9. Miltiadis Allamanis, Pankajan Chanthirasegaran, Pushmeet Kohli, and Charles Sutton. 2017. Learning Continuous Semantic Representations of Symbolic Expressions. In Proceedings of the International Conference on Machine Learning (ICML). find
  10. Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A Convolutional Attention Network for Extreme Summarization of Source Code. In Proceedings of the International Conference on Machine Learning (ICML).
  11. Miltiadis Allamanis and Charles Sutton. 2013. Mining source code repositories at massive scale using language modeling. In Proceedings of the Working Conference on Mining Software Repositories (MSR). find
  12. Miltiadis Allamanis and Charles Sutton. 2014. Mining idioms from source code. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  13. Miltiadis Allamanis, Daniel Tarlow, Andrew Gordon, and Yi Wei. 2015. Bimodal modelling of source code and natural language. In Proceedings of the International Conference on Machine Learning (ICML). find
  14. Sven Amann, Sebastian Proksch, Sarah Nadi, and Mira Mezini. 2016. A study of Visual Studio usage in practice. In Proceedings of the International Conference on Software Analysis, Evolution, and Reengineering (SANER). find
  15. Gene M Amdahl. 1967. Validity of the single processor approach to achieving large scale computing capabilities. In Proceedings of the April 18-20, 1967, spring joint computer conference. find
  16. Matthew Amodio, Swarat Chaudhuri, and Thomas Reps. 2017. Neural Attribute Machines for Program Generation. arXiv preprint arXiv:1705.09231 (2017). Link link link link link link link link link link link link link find
  17. Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016. Learning to Compose Neural Networks for Question Answering. In Proceedings of NAACL-HLT.
  18. Daniel Arp, Michael Spreitzenbarth, Malte Hubner, Hugo Gascon, and Konrad Rieck. 2014. DREBIN: Effective and Explainable Detection of Android Malware in Your Pocket. In Network and Distributed System Security Symposium. find
  19. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations (ICLR).
  20. Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2017. DeepCoder: Learning to Write Programs. In Proceedings of the International Conference on Learning Representations (ICLR). find
  21. Antonio Valerio Miceli Barone and Rico Sennrich. 2017. A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. arXiv preprint arXiv:1707.02275 (2017). find
  22. Rohan Bavishi, Michael Pradel, and Koushik Sen. 2017. Context2Name: A Deep Learning-Based Approach to Infer Natural Variable Names from Usage Contexts. (2017). find
  23. Tony Beltramelli. 2017. pix2code: Generating Code from a Graphical User Interface Screenshot. arXiv preprint arXiv:1705.07962 (2017). — [arXiv:1705.07962](https://arxiv.org/abs/1705.07962) find
  24. Al Bessey, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri-Gros, Asya Kamsky, Scott McPeak, and Dawson Engler. 2010. A few billion lines of code later: using static analysis to find bugs in the real world. Commun. ACM (2010). find
  25. Sahil Bhatia and Rishabh Singh. 2018. Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural Networks. In Proceedings of the International Conference on Software Engineering (ICSE). find
  26. Avishkar Bhoopchand, Tim Rocktäschel, Earl Barr, and Sebastian Riedel. 2016. Learning Python Code Suggestion with a Sparse Pointer Network. arXiv preprint arXiv:1611.08307 (2016). find
  27. Benjamin Bichsel, Veselin Raychev, Petar Tsankov, and Martin Vechev. 2016. Statistical Deobfuscation of Android Applications. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. find
  28. Pavol Bielik, Veselin Raychev, and Martin Vechev. 2015. Programming with “Big Code”: Lessons, Techniques and Applications. In LIPIcs-Leibniz International Proceedings in Informatics. find
  29. Pavol Bielik, Veselin Raychev, and Martin Vechev. 2016. PHOG: Probabilistic Model for Code. In Proceedings of the International Conference on Machine Learning (ICML). find
  30. David M Blei. 2012. Probabilistic Topic Models. Commun. ACM (2012). find
  31. Marc Brockschmidt, Yuxin Chen, Pushmeet Kohli, Siddharth Krishna, and Daniel Tarlow. 2017. Learning Shape Analysis. In International Static Analysis Symposium. Springer. find
  32. Peter John Brown. 1979. Software Portability: an advanced course. CUP Archive. find
  33. Marcel Bruch, Martin Monperrus, and Mira Mezini. 2009. Learning from examples to improve code completion systems. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FSE). find
  34. Raymond PL Buse and Westley Weimer. 2012. Synthesizing API usage examples. In Proceedings of the International Conference on Software Engineering (ICSE). find
  35. Joshua Charles Campbell, Abram Hindle, and José Nelson Amaral. 2014. Syntax errors just aren’t natural: improving error reporting with language models. In Proceedings of the Working Conference on Mining Software Repositories (MSR). find
  36. Lei Cen, Christoher S Gates, Luo Si, and Ninghui Li. 2015. A probabilistic discriminative model for android malware detection with decompiled source code. IEEE Transactions on Dependable and Secure Computing (2015). find
  37. Luigi Cerulo, Massimiliano Di Penta, Alberto Bacchelli, Michele Ceccarelli, and Gerardo Canfora. 2015. Irish: A Hidden Markov Model to detect coded information islands in free text. Science of Computer Programming (2015). find
  38. Kwonsoo Chae, Hakjoo Oh, Kihong Heo, and Hongseok Yang. 2017. Automatically generating features for learning program analysis heuristics for C-like languages. Proceedings of the Conference on Object-Oriented Programming, Systems, Languages & Applications (OOPSLA) (2017). find
  39. Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM Computing Surveys (CSUR) (2009). find
  40. Stanley F Chen and Joshua Goodman. 1999. An empirical study of smoothing techniques for language modeling. Computer Speech and Language (1999). find
  41. Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder–Decoder Approaches. Syntax, Semantics and Structure in Statistical Translation (2014).
  42. Edmund Clarke, Daniel Kroening, and Karen Yorav. 2003. Behavioral consistency of C and Verilog programs using bounded model checking. In Proceedings of the 40th annual Design Automation Conference. find
  43. Trevor Cohn, Phil Blunsom, and Sharon Goldwater. 2010. Inducing tree-substitution grammars. Journal of Machine Learning Research (JMLR) (2010). find
  44. Christopher S Corley, Kostadin Damevski, and Nicholas A Kraft. 2015. Exploring the use of deep learning for feature location. In Software Maintenance and Evolution (ICSME), 2015 IEEE International Conference on. find
  45. Patrick Cousot, Radhia Cousot, Jerôme Feret, Laurent Mauborgne, Antoine Miné, David Monniaux, and Xavier Rival. 2005. The ASTRÉE analyzer. In ESPO. Springer. find
  46. William Croft. 2008. Evolutionary linguistics. Annual Review of Anthropology (2008). find
  47. Chris Cummins, Pavlos Petoumenos, Zheng Wang, and Hugh Leather. 2017a. End-to-end Deep Learning of Optimization Heuristics. In International Conference on Parallel Computing Technologies. find
  48. Chris Cummins, Pavlos Petoumenos, Zheng Wang, and Hugh Leather. 2017b. Synthesizing benchmarks for predictive modeling. In IEEE/ACM International Symposium on Code Generation and Optimization (CGO). find
  49. Hoa Khanh Dam, Truyen Tran, and Trang Pham. 2016. A deep language model for software code. arXiv preprint arXiv:1608.02715 (2016). — [arXiv:1608.02715](https://arxiv.org/abs/1608.02715) find
  50. Florian Deißenböck and Markus Pizka. 2006. Concise and consistent naming. Software Quality Journal (2006). find
  51. Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M Rush. 2017. Image-to-Markup Generation with Coarse-to-Fine Attention. In Proceedings of the International Conference on Machine Learning (ICML). find
  52. Premkumar Devanbu. 2015. New initiative: the naturalness of software. In Proceedings of the International Conference on Software Engineering (ICSE). find
  53. Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel rahman Mohamed, and Pushmeet Kohli. 2017. RobustFill: Neural Program Learning under Noisy I/O. In Proceedings of the International Conference on Machine Learning (ICML). find
  54. Robert Dyer, Hoan Anh Nguyen, Hridesh Rajan, and Tien N. Nguyen. 2013. Boa: A Language and Infrastructure for Analyzing Ultra-Large-Scale Software Repositories. In Proceedings of the International Conference on Software Engineering (ICSE). find
  55. Kevin Ellis, Daniel Ritchie, Armando Solar-Lezama, and Joshua B Tenenbaum. 2017. Learning to Infer Graphics Programs from Hand-Drawn Images. arXiv preprint arXiv:1707.09627 (2017). find
  56. Dawson Engler, David Yu Chen, Seth Hallem, Andy Chou, and Benjamin Chelf. 2001. Bugs as deviant behavior: A general approach to inferring errors in systems code. In ACM SIGOPS Operating Systems Review. find
  57. Michael D Ernst. 2017. Natural language is a programming language: Applying natural language processing to software development. In LIPIcs-Leibniz International Proceedings in Informatics. find
  58. Ethan Fast, Daniel Steffee, Lucy Wang, Joel R Brandt, and Michael S Bernstein. 2014. Emergent, crowd-scale programming practice in the IDE. In Proceedings of the Annual ACM Conference on Human Factors in Computing Systems. find
  59. John K Feser, Marc Brockschmidt, Alexander L Gaunt, and Daniel Tarlow. 2017. Neural functional programming. 2016. Proceedings of the International Conference on Learning Representations (ICLR) (2017). find
  60. Matthew Finifter, Adrian Mettler, Naveen Sastry, and David Wagner. 2008. Verifiable functional purity in java. In Proceedings of the 15th ACM conference on Computer and communications security. ACM, 161–174. find
  61. Eclipse Foundation. Code Recommenders. www.eclipse.org/recommenders. (????). Visited June 2017. find
  62. Jaroslav Fowkes, Pankajan Chanthirasegaran, Razvan Ranca, Miltos Allamanis, Mirella Lapata, and Charles Sutton. 2017. Autofolding for Source Code Summarization. IEEE Transactions on Software Engineering (TSE) (2017). find
  63. Jaroslav Fowkes and Charles Sutton. 2015. Parameter-Free Probabilistic API Mining at GitHub Scale. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  64. Christine Franks, Zhaopeng Tu, Premkumar Devanbu, and Vincent Hellendoorn. 2015. Cacheca: A cache language model based code suggestion tool. In Proceedings of the International Conference on Software Engineering (ICSE). find
  65. Wei Fu and Tim Menzies. 2017. Easy over Hard: A Case Study on Deep Learning. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  66. Mark Gabel and Zhendong Su. 2008. Javert: fully automatic mining of general temporal properties from dynamic traces. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  67. Mark Gabel and Zhendong Su. 2010. A study of the uniqueness of source code. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  68. Rosalva E Gallardo-Valencia and Susan Elliott Sim. 2009. Internet-scale code search. In Proceedings of the 2009 ICSE Workshop on Search-Driven Development-Users, Infrastructure, Tools and Evaluation. find
  69. Alexander L Gaunt, Marc Brockschmidt, Rishabh Singh, Nate Kushman, Pushmeet Kohli, Jonathan Taylor, and Daniel Tarlow. 2016. TerpreT: A probabilistic programming language for program induction. arXiv preprint arXiv:1608.04428 (2016). find
  70. Spandana Gella, Mirella Lapata, and Frank Keller. 2016. Unsupervised Visual Sense Disambiguation for Verbs using Multimodal Embeddings. In Proceedings of NAACL-HLT. find
  71. Elena L Glassman, Jeremy Scott, Rishabh Singh, Philip J Guo, and Robert C Miller. 2015. OverCode: Visualizing variation in student solutions to programming problems at scale. ACM Transactions on Computer-Human Interaction (TOCHI) (2015). find
  72. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. www.deeplearningbook.org. find
  73. Andrew D Gordon, Thomas A Henzinger, Aditya V Nori, and Sriram K Rajamani. 2014. Probabilistic programming. In Proceedings of the International Conference on Software Engineering (ICSE). find
  74. Orlena Gotel, Jane Cleland-Huang, Jane Huffman Hayes, Andrea Zisman, Alexander Egyed, Paul Grünbacher, Alex Dekhtyar, Giuliano Antoniol, Jonathan Maletic, and Patrick Mäder. 2012. Traceability fundamentals. In Software and Systems Traceability. Springer. find
  75. Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural Turing Machines. arXiv preprint arXiv:1410.5401 (2014). find
  76. Xiaodong Gu, Hongyu Zhang, Dongmei Zhang, and Sunghun Kim. 2016. Deep API Learning. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  77. Sumit Gulwani and Mark Marron. 2014. NLyze: Interactive programming by natural language for spreadsheet data analysis and manipulation. In Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data. find
  78. Sumit Gulwani, Oleksandr Polozov, Rishabh Singh, and others. 2017. Program Synthesis. Foundations and Trends® in Programming Languages (2017).
  79. Jin Guo, Jinghui Cheng, and Jane Cleland-Huang. 2017. Semantically enhanced software traceability using deep learning techniques. In Proceedings of the International Conference on Software Engineering (ICSE). find
  80. Rahul Gupta, Aditya Kanade, and Shirish Shevade. 2018. Deep Reinforcement Learning for Programming Language Correction. arXiv preprint arXiv:1801.10467 (2018). — [arXiv:1801.10467](https://arxiv.org/abs/1801.10467) find
  81. Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish Shevade. 2017. DeepFix: Fixing common C language errors by deep learning. In Proceedings of the Conference of Artificial Intelligence (AAAI). find
  82. Tihomir Gvero and Viktor Kuncak. 2015. Synthesizing Java expressions from free-form queries. In Proceedings of the Conference on Object-Oriented Programming, Systems, Languages & Applications (OOPSLA). find
  83. Alon Halevy, Peter Norvig, and Fernando Pereira. 2009. The unreasonable effectiveness of data. IEEE Intelligent Systems (2009). find
  84. Vincent J Hellendoorn and Premkumar Devanbu. 2017. Are deep neural networks the best choice for modeling source code?. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  85. Vincent J Hellendoorn, Premkumar T Devanbu, and Alberto Bacchelli. 2015. Will they like this?: Evaluating code contributions with language models. In Proceedings of the Working Conference on Mining Software Repositories (MSR). find
  86. Felix Hill, KyungHyun Cho, Anna Korhonen, and Yoshua Bengio. 2016. Learning to Understand Phrases by Embedding the Dictionary. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) (2016).
  87. Abram Hindle, Earl T Barr, Mark Gabel, Zhendong Su, and Premkumar Devanbu. 2016. On the naturalness of software. Commun. ACM (2016). find
  88. Abram Hindle, Earl T Barr, Zhendong Su, Mark Gabel, and Premkumar Devanbu. 2012. On the naturalness of software. In Proceedings of the International Conference on Software Engineering (ICSE). find
  89. Geoffrey E Hinton. 1984. Distributed representations. (1984). find
  90. C. A. R. Hoare. 1969. An Axiomatic Basis for Computer Programming. Commun. ACM 12, 10 (Oct. 1969), 576–580. DOI:http://dx.doi.org/10.1145/363235.363259 — [doi:10.1145/363235.363259](https://doi.org/10.1145/363235.363259) find
  91. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation (1997). find
  92. Reid Holmes, Robert J Walker, and Gail C Murphy. 2005. Strathcona example recommendation tool. In ACM SIGSOFT Software Engineering Notes. find
  93. Chun-Hung Hsiao, Michael Cafarella, and Satish Narayanasamy. 2014. Using web corpus statistics for program analysis. In ACM SIGPLAN Notices. find
  94. Xing Hu, Yuhan Wei, Ge Li, and Zhi Jin. 2017. CodeSum: Translate Program Language to Natural Language. arXiv preprint arXiv:1708.01837 (2017). — [arXiv:1708.01837](https://arxiv.org/abs/1708.01837) find
  95. Andrew Hunt and David Thomas. 2000. The pragmatic programmer: from journeyman to master. Addison-Wesley Professional. find
  96. Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2016. Summarizing Source Code using a Neural Attention Model. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  97. Siyuan Jiang, Ameer Armaly, and Collin McMillan. 2017. Automatically generating commit messages from diffs using neural machine translation. In Proceedings of the International Conference on Automated Software Engineering (ASE). find
  98. Daniel D Johnson. 2016. Learning graphical state transitions. In Proceedings of the International Conference on Learning Representations (ICLR). find
  99. Dan Jurafsky. 2000. Speech & Language Processing (3 ed.). Pearson Education. find
  100. René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA). find
  101. Neel Kant. 2018. Recent Advances in Neural Program Synthesis. arXiv preprint arXiv:1802.02353 (2018). — [arXiv:1802.02353](https://arxiv.org/abs/1802.02353) find
  102. Svetoslav Karaivanov, Veselin Raychev, and Martin Vechev. 2014. Phrase-based statistical translation of programming languages. In International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software. find
  103. Andrej Karpathy, Justin Johnson, and Fei-Fei Li. 2015. Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078 (2015).
  104. Reinhard Kneser and Hermann Ney. 1995. Improved backing-off for m-gram language modeling. In Acoustics, Speech, and Signal Processing, 1995. ICASSP-95., 1995 International Conference on. find
  105. Donald E Knuth. 1984. Literate programming. Comput. J. (1984). find
  106. Ugur Koc, Parsa Saadatpanah, Jeffrey S Foster, and Adam A Porter. 2017. Learning a classifier for false positive error reports emitted by static code analysis tools. In Proceedings of the 1st ACM SIGPLAN International Workshop on Machine Learning and Programming Languages. find
  107. Rainer Koschke. 2007. Survey of research on software clones. In Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik. find
  108. Ted Kremenek, Andrew Y Ng, and Dawson R Engler. 2007. A Factor Graph Model for Software Bug Finding. In Proceedings of the International Joint Conference on Artifical intelligence (IJCAI). find
  109. Roland Kuhn and Renato De Mori. 1990. A cache-based natural language model for speech recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on (1990). find
  110. Nate Kushman and Regina Barzilay. 2013. Using Semantic Unification to Generate Regular Expressions from Natural Language. In Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
  111. Tessa Lau. 2001. Programming by demonstration: a machine learning approach. Ph.D. Dissertation. University of Washington. find
  112. Quoc V Le and Tomas Mikolov. 2014. Distributed Representations of Sentences and Documents. In Proceedings of the International Conference on Machine Learning (ICML).
  113. Tien-Duy B Le, Mario Linares-Vásquez, David Lo, and Denys Poshyvanyk. 2015. Rclinker: Automated linking of issue reports and commits leveraging rich contextual information. In Proceedings of the International Conference on Program Comprehension (ICPC). find
  114. Dor Levy and Lior Wolf. 2017. Learning to Align the Source Code to the Compiled Object Code. In Proceedings of the International Conference on Machine Learning (ICML). find
  115. Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2016. Gated Graph Sequence Neural Networks. Proceedings of the International Conference on Learning Representations (ICLR) (2016). find
  116. Percy Liang, Michael I Jordan, and Dan Klein. 2010. Learning programs: A hierarchical Bayesian approach. In Proceedings of the International Conference on Machine Learning (ICML). find
  117. Ben Liblit, Mayur Naik, Alice X Zheng, Alex Aiken, and Michael I Jordan. 2005. Scalable statistical bug isolation. In ACM SIGPLAN Notices. find
  118. Xi Victoria Lin, Chenglong Wang, Deric Pang, Kevin Vu, Luke Zettlemoyer, and Michael D. Ernst. 2017. Program synthesis from natural language using recurrent neural networks. Technical Report UW-CSE-17-03-01. University of Washington Department of Computer Science and Engineering, Seattle, WA, USA. find
  119. Xi Victoria Lin, Chenglong Wang, Luke Zettlemoyer, and Michael D. Ernst. 2018. NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System. In International Conference on Language Resources and Evaluation. find
  120. Wang Ling, Edward Grefenstette, Karl Moritz Hermann, Tomas Kocisky, Andrew Senior, Fumin Wang, and Phil Blunsom. 2016. Latent Predictor Networks for Code Generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  121. Han Liu. 2016. Towards better program obfuscation: optimization via language models. In Proceedings of the 38th International Conference on Software Engineering Companion. find
  122. Benjamin Livshits, Aditya V. Nori, Sriram K. Rajamani, and Anindya Banerjee. 2009. Merlin: Specification Inference for Explicit Information Flow Problems. In Proceedings of the Symposium on Programming Language Design and Implementation (PLDI). find
  123. Sarah M. Loos, Geoffrey Irving, Christian Szegedy, and Cezary Kaliszyk. 2017. Deep Network Guided Proof Search. In LPAR.
  124. Pablo Loyola, Edison Marrese-Taylor, and Yutaka Matsuo. 2017. A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes. arXiv preprint arXiv:1704.04856 (2017). find
  125. Yanxin Lu, Swarat Chaudhuri, Chris Jermaine, and David Melski. 2017. Data-Driven Program Completion. arXiv preprint arXiv:1705.09042 (2017). — [arXiv:1705.09042](https://arxiv.org/abs/1705.09042) find
  126. Chris Maddison and Daniel Tarlow. 2014. Structured Generative Models of Natural Source Code. In Proceedings of the International Conference on Machine Learning (ICML). find
  127. Ravi Mangal, Xin Zhang, Aditya V Nori, and Mayur Naik. 2015. A user-guided approach to program analysis. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  128. Collin Mcmillan, Denys Poshyvanyk, Mark Grechanik, Qing Xie, and Chen Fu. 2013. Portfolio: Searching for relevant functions and their usages in millions of lines of code. ACM Transactions on Software Engineering and Methodology (TOSEM) (2013). find
  129. Aditya Menon, Omer Tamuz, Sumit Gulwani, Butler Lampson, and Adam Kalai. 2013. A machine learning framework for programming by example. In Proceedings of the International Conference on Machine Learning (ICML). find
  130. Kim Mens and Angela Lozano. 2014. Source code-based recommendation systems. In Recommendation Systems in Software Engineering. Springer. find
  131. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).
  132. Lili Mou, Ge Li, Lu Zhang, Tao Wang, and Zhi Jin. 2016. Convolutional Neural Networks over Tree Structures for Programming Language Processing. In Proceedings of the Conference of Artificial Intelligence (AAAI). find
  133. Dana Movshovitz-Attias and William W. Cohen. 2013. Natural Language Models for Predicting Programming Comments. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  134. Dana Movshovitz-Attias and William W. Cohen. 2015. KB-LDA: Jointly learning a knowledge base of hierarchy, relations, and facts. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  135. Vijayaraghavan Murali, Swarat Chaudhuri, and Chris Jermaine. 2017a. Bayesian Sketch Learning for Program Synthesis. arXiv preprint arXiv:1703.05698 (2017). find
  136. Vijayaraghavan Murali, Swarat Chaudhuri, and Chris Jermaine. 2017b. Finding Likely Errors with Bayesian Specifications. arXiv preprint arXiv:1703.01370 (2017). — [arXiv:1703.01370](https://arxiv.org/abs/1703.01370) find
  137. Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. 2015. Neural Programmer: Inducing latent programs with gradient descent. In Proceedings of the International Conference on Learning Representations (ICLR). find
  138. Graham Neubig. 2016. Survey of Methods to Generate Natural Language from Source Code. http://www.languageandcode.org/ nlse2015/ neubig15nlse-survey.pdf (2016). ↗
  139. Anh Tuan Nguyen and Tien N. Nguyen. 2015. Graph-based statistical language model for code. In Proceedings of the International Conference on Software Engineering (ICSE). find
  140. Anh Tuan Nguyen, Tung Thanh Nguyen, and Tien N Nguyen. 2013. Lexical statistical machine translation for language migration. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  141. Anh T Nguyen, Tung Thanh Nguyen, and Tien N. Nguyen. 2015. Divide-and-Conquer Approach for Multi-phase Statistical Migration for Source Code. In Proceedings of the International Conference on Automated Software Engineering (ASE). find
  142. Trong Duc Nguyen, Anh Tuan Nguyen, and Tien N Nguyen. 2016. Mapping API elements for code migration with vector representations. In Proceedings of the International Conference on Software Engineering (ICSE). find
  143. Trong Duc Nguyen, Anh Tuan Nguyen, Hung Dang Phan, and Tien N Nguyen. 2017. Exploring API embedding for API usages and applications. In Proceedings of the International Conference on Software Engineering (ICSE). find
  144. Tung Thanh Nguyen, Anh Tuan Nguyen, Hoan Anh Nguyen, and Tien N Nguyen. 2013. A statistical semantic language model for source code. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FSE). find
  145. Haoran Niu, Iman Keivanloo, and Ying Zou. 2016. Learning to rank code examples for code search engines. Empirical Software Engineering (ESEM) (2016). find
  146. Yusuke Oda, Hiroyuki Fudaba, Graham Neubig, Hideaki Hata, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2015. Learning to Generate Pseudo-Code from Source Code Using Statistical Machine Translation. In Proceedings of the International Conference on Automated Software Engineering (ASE). find
  147. Hakjoo Oh, Hongseok Yang, and Kwangkeun Yi. 2015. Learning a strategy for adapting a program analysis via Bayesian optimisation. In Proceedings of the Conference on Object-Oriented Programming, Systems, Languages & Applications (OOPSLA). find
  148. Cyrus Omar. 2013. Structured statistical syntax tree prediction. In Proceedings of the Conference on Systems, Programming, Languages and Applications: Software for Humanity (SPLASH). find
  149. Cyrus Omar, Ian Voysey, Michael Hilton, Joshua Sunshine, Claire Le Goues, Jonathan Aldrich, and Matthew A Hammer. 2017. Toward Semantic Foundations for Program Editors. (2017). find
  150. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  151. Emilio Parisotto, Abdel-rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. 2017. Neuro-Symbolic Program Synthesis. In Proceedings of the International Conference on Learning Representations (ICLR).
  152. Terence Parr and Jurgen J. Vinju. 2016. Towards a universal code formatter through machine learning. In SLE. find
  153. Jibesh Patra and Michael Pradel. 2016. Learning to Fuzz: Application-Independent Fuzz Testing with Probabilistic, Generative Models of Input Data. (2016). find
  154. Hung Viet Pham, Phong Minh Vu, Tung Thanh Nguyen, and others. 2016. Learning API usages from bytecode: a statistical approach. In Proceedings of the International Conference on Software Engineering (ICSE). find
  155. Chris Piech, Jonathan Huang, Andy Nguyen, Mike Phulsuksombati, Mehran Sahami, and Leonidas J Guibas. 2015. Learning Program Embeddings to Propagate Feedback on Student Code. In Proceedings of the International Conference on Machine Learning (ICML). find
  156. Matt Post and Daniel Gildea. 2009. Bayesian learning of a tree substitution grammar. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  157. Michael Pradel and Koushik Sen. 2017. Deep Learning to Find Bugs. (2017). find
  158. Sebastian Proksch, Sven Amann, Sarah Nadi, and Mira Mezini. 2016. Evaluating the evaluations of code recommender systems: a reality check. In Proceedings of the International Conference on Automated Software Engineering (ASE). find
  159. Sebastian Proksch, Johannes Lerch, and Mira Mezini. 2015. Intelligent code completion with Bayesian networks. ACM Transactions on Software Engineering and Methodology (TOSEM) (2015). find
  160. Yewen Pu, Karthik Narasimhan, Armando Solar-Lezama, and Regina Barzilay. 2016. sk_p: a neural program corrector for MOOCs. In Proceedings of the Conference on Systems, Programming, Languages and Applications: Software for Humanity (SPLASH). find
  161. Chris Quirk, Raymond Mooney, and Michel Galley. 2015. Language To Code: Learning Semantic Parsers for If-This-Then-That Recipes. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  162. Maxim Rabinovich, Mitchell Stern, and Dan Klein. 2017. Abstract Syntax Networks for Code Generation and Semantic Parsing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  163. Baishakhi Ray, Vincent Hellendoorn, Saheel Godhane, Zhaopeng Tu, Alberto Bacchelli, and Premkumar Devanbu. 2016. On the naturalness of buggy code. In Proceedings of the International Conference on Software Engineering (ICSE). find
  164. Veselin Raychev, Pavol Bielik, Martin Vechev, and Andreas Krause. 2016. Learning programs from noisy data. In Proceedings of the Symposium on Principles of Programming Languages (POPL). find
  165. Veselin Raychev, Martin Vechev, and Andreas Krause. 2015. Predicting Program Properties from “Big Code”. In Proceedings of the Symposium on Principles of Programming Languages (POPL). find
  166. Veselin Raychev, Martin Vechev, and Eran Yahav. 2014. Code completion with statistical language models. In Proceedings of the Symposium on Programming Language Design and Implementation (PLDI). find
  167. Scott Reed and Nando de Freitas. 2016. Neural Programmer-Interpreters. In Proceedings of the International Conference on Learning Representations (ICLR). find
  168. Sebastian Riedel, Matko Bosnjak, and Tim Rocktäschel. 2017. Programming with a Differentiable Forth Interpreter. In ICML. find
  169. Martin Robillard, Robert Walker, and Thomas Zimmermann. 2010. Recommendation systems for software engineering. Software, IEEE (2010). find
  170. Martin P Robillard, Walid Maalej, Robert J Walker, and Thomas Zimmermann. 2014. Recommendation systems in software engineering. Springer. find
  171. Tim Rocktäschel and Sebastian Riedel. 2017. End-to-end Differentiable Proving. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS).
  172. Caitlin Sadowski, Kathryn T Stolee, and Sebastian Elbaum. 2015. How developers search for code: a case study. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  173. Juliana Saraiva, Christian Bird, and Thomas Zimmermann. 2015. Products, developers, and milestones: how should I build my N-Gram language model. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  174. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  175. Abhishek Sharma, Yuan Tian, and David Lo. 2015. NIRMAL: Automatic identification of software relevant tweets leveraging language model. In Proceedings of the International Conference on Software Analysis, Evolution, and Reengineering (SANER). find
  176. Rishabh Singh and Sumit Gulwani. 2015. Predicting a correct program in programming by example. In International Conference on Computer Aided Verification. find
  177. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS).
  178. Suresh Thummalapenta and Tao Xie. 2007. Parseweb: a programmer assistant for reusing open source code on the web. In Proceedings of the International Conference on Automated Software Engineering (ASE). find
  179. Christoph Treude and Martin P Robillard. 2016. Augmenting API documentation with insights from Stack Overflow. In Proceedings of the International Conference on Software Engineering (ICSE). find
  180. Zhaopeng Tu, Zhendong Su, and Premkumar Devanbu. 2014. On the localness of software. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  181. Bogdan Vasilescu, Casey Casalnuovo, and Premkumar Devanbu. 2017. Recovering clear, natural identifiers from obfuscated JS names. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE). find
  182. Lisa Wang, Angela Sy, Larry Liu, and Chris Piech. 2017. Deep Knowledge Tracing On Programming Exercises. In Conference on Learning @ Scale. find
  183. Song Wang, Devin Chollak, Dana Movshovitz-Attias, and Lin Tan. 2016a. Bugram: bug detection with n-gram language models. In Proceedings of the International Conference on Automated Software Engineering (ASE). find
  184. Song Wang, Taiyue Liu, and Lin Tan. 2016b. Automatically learning semantic features for defect prediction. In Proceedings of the International Conference on Software Engineering (ICSE). find
  185. Xin Wang, Chang Liu, Richard Shin, Joseph E. Gonzalez, and Dawn Song. 2016c. Neural Code Completion. (2016). https://openreview.net/pdf?id=rJbPBt9lg. ↗
  186. Andrzej Wasylkowski, Andreas Zeller, and Christian Lindig. 2007. Detecting object usage anomalies. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the Symposium on the Foundations of Software Engineering (ESEC/FSE). find
  187. Martin White, Michele Tufano, Christopher Vendome, and Denys Poshyvanyk. 2016. Deep Learning Code Fragments for Code Clone Detection. In Proceedings of the International Conference on Automated Software Engineering (ASE). find
  188. Martin White, Christopher Vendome, Mario Linares-Vásquez, and Denys Poshyvanyk. 2015. Toward deep learning software repositories. In Proceedings of the Working Conference on Mining Software Repositories (MSR). find
  189. Chadd C Williams and Jeffrey K Hollingsworth. 2005. Automatic mining of source code repositories to improve bug finding techniques. IEEE Transactions on Software Engineering (TSE) (2005). find
  190. Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. find
  191. W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering (TSE) (2016). find
  192. Tao Xie and Jian Pei. 2006. MAPO: Mining API usages from open source repositories. In Proceedings of the Working Conference on Mining Software Repositories (MSR). find
  193. Chang Xu, Dacheng Tao, and Chao Xu. 2013. A survey on multi-view learning. arXiv preprint arXiv:1304.5634 (2013). — [arXiv:1304.5634](https://arxiv.org/abs/1304.5634) find
  194. Shir Yadid and Eran Yahav. 2016. Extracting code from programming tutorial videos. In Proceedings of the 2016 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software. find
  195. Eran Yahav. 2015. Programming with “Big Code”. In Asian Symposium on Programming Languages and Systems. Springer, 3–8. find
  196. Pengcheng Yin and Graham Neubig. 2017. A Syntactic Neural Model for General-Purpose Code Generation. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) (2017).
  197. Wojciech Zaremba and Ilya Sutskever. 2014. Learning to execute. arXiv preprint arXiv:1410.4615 (2014). find
  198. Alice X Zheng, Michael I Jordan, Ben Liblit, and Alex Aiken. 2003. Statistical debugging of sampled programs. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS). find
  199. Alice X Zheng, Michael I Jordan, Ben Liblit, Mayur Naik, and Alex Aiken. 2006. Statistical debugging: simultaneous identification of multiple bugs. In Proceedings of the International Conference on Machine Learning (ICML). find
  200. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. arXiv preprint arXiv:1709.00103 (2017).
  201. Thomas Zimmermann, Andreas Zeller, Peter Weissgerber, and Stephan Diehl. 2005. Mining version histories to guide software changes. IEEE Transactions on Software Engineering (TSE) (2005). find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/allamanis-2017-survey-machine-learning-big-code/