Elamur Research⌕
☰
home / papers / 2018 / Mapping Language to Code in Programmatic Context

Mapping Language to Code in Programmatic Context

Paper · 2018 · Srinivasan Iyer, Ioannis Konstas, Alvin Cheung et al.

Summary

Real programs depend on their surroundings, yet earlier NL-to-code systems used fixed templates, one fixed context, or no context. This paper defines a new task: write a Java class member function from its English documentation while taking into account the rest of the class, namely its member variables with their types and its other methods with their return types. The authors built CONCODE, a dataset of more than 100,000 (class environment, NL, code) tuples mined from documented Java files in public GitHub repositories. It covers thousands of domains, so models meet unseen identifiers at test time. Their encoder-decoder model encodes the documentation and the environment identifiers with Bi-LSTMs, splitting names into camel-case pieces. The decoder produces grammar production rules, which guarantees syntactically valid code. It uses two-stage attention, first over the text and then over the environment, and it copies class variables and methods through a copying component trained with supervision. The model reaches 8.6% exact match and 22.11 BLEU, beating retrieval and neural baselines. This shows that programmatic context helps, though the task is far from solved.

Key points

  • CONCODE dataset: more than 100,000 Java examples from GitHub. This is about ten times larger than MTG (Ling et al., 2016; 13k examples), and it is the first such dataset to condition generation on the code environment.
  • Environment encoding: variable and method names are split by camel-casing into subword units. Each name is encoded with a Bi-LSTM and then contextualized together with its type by another Bi-LSTM.
  • Decoder design:
    • It builds the AST derivation depth-first, left to right, starting from MemberDeclaration, similar to Yin and Neubig (2017).
    • Its two-step attention first attends over the NL, then over the environment.
    • Environment identifiers are reproduced through a copying component trained with direct supervision, following Gu et al. (2016a).
  • Results: 8.6% exact match accuracy and 22.11 BLEU, a 1.95% exact-match gain over strong neural baselines.
  • Analysis: ablations separate the contributions of context and model design. The error analysis suggests that richer context, such as method parameters and bodies, could improve results further.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WashingtonSeattle · United States402 works · 212,762 citations🎓Heriot-Watt UniversityEdinburgh · United Kingdom17 works · 5,994 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

Cite

Cite as Srinivasan Iyer, Ioannis Konstas, Alvin Cheung and Luke Zettlemoyer. 2018. Mapping Language to Code in Programmatic Context. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1192 · arXiv 1808.09588

@inproceedings{iyer2018mapping,
  title = {{Mapping Language to Code in Programmatic Context}},
  author = {Srinivasan Iyer, Ioannis Konstas, Alvin Cheung and Luke Zettlemoyer},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1192},
  eprint = {1808.09588},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d18-1192}
}

References

20 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Miltiadis Allamanis, Earl T Barr, Christian Bird, and Charles Sutton. 2015a. Suggesting accurate method and class names. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, pages 38–49. find
  2. Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A convolutional attention network for extreme summarization of source code. In International Conference on Machine Learning, pages 2091–2100.
  3. Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Jayant Krishnamurthy, and Luke Zettlemoyer. 2017. Learning a neural semantic parser from user feedback. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 963–973, Vancouver, Canada… find
  4. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.
  5. Jayant Krishnamurthy, Pradeep Dasigi, and Matt Gardner. 2017. Neural semantic parsing with type constraints for semi-structured tables. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1516–1526, Copenhagen, Denmark. Association for Computational Lingu… find
  6. Nate Kushman and Regina Barzilay. 2013. Using semantic unification to generate regular expressions from natural language. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 826–836. Association for… find
  7. Percy Liang, I. Michael Jordan, and Dan Klein. 2013. Learning dependency-based compositional semantics. Computational Linguistics, 39(2). find
  8. Wang Ling, Phil Blunsom, Edward Grefenstette, Moritz Karl Hermann, Tomáš Kočiský, Fumin Wang, and Andrew Senior. 2016. Latent predictor networks for code generation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 599–609.…
  9. Nicholas Locascio, Karthik Narasimhan, Eduardo De Leon, Nate Kushman, and Regina Barzilay. 2016. Neural generation of regular expressions from natural language with minimal domain knowledge. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1918–1923, A… find
  10. Thang Luong, Hieu Pham, and D. Christopher Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421. Association for Computational Linguistics. the 40th annual meeting…
  11. Chris Quirk, Raymond Mooney, and Michel Galley. 2015. Language to code: Learning semantic parsers for if-this-then-that recipes. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volum…
  12. Maxim Rabinovich, Mitchell Stern, and Dan Klein. 2017. Abstract syntax networks for code generation and semantic parsing. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1139– 1149, Vancouver, Canada. Association for Computati… find
  13. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Li…
  14. Pengcheng Yin and Graham Neubig. 2017. A syntactic neural model for general-purpose code generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 440–450, Vancouver, Canada. Association for Computational Linguistics.
  15. Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: structured classification with probabilistic categorial grammars. In UAI ’05, Proceedings of the 21st Conference in Uncertainty in Artificial Intelligence.
  16. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103.
  17. Mehdi Hafezi Manshadi, Daniel Gildea, and James F Allen. 2013. Integrating programming by example and natural language programming. In AAAI. find
  18. Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. 2015. Neural programmer: Inducing latent programs with gradient descent. arXiv preprint arXiv:1511.04834. find
  19. Yusuke Oda, Hiroyuki Fudaba, Graham Neubig, Hideaki Hata, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. 2015. Learning to generate pseudo-code from source code using statistical machine translation (t). In Automated Software Engineering (ASE), 2015 30th IEEE/ACM International Conference on, pag… find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/iyer-2018-mapping-language-code-programmatic-context/