Elamur Research⌕
☰
home / papers / 2024 / CoCoMIC: Code Completion by Jointly Modeling In-file and Cross-file Context

CoCoMIC: Code Completion by Jointly Modeling In-file and Cross-file Context

Paper · 2024 · Yangruibo Ding, Zijian Wang, Wasi Ahmad et al.

Summary

Most code language models complete a file using only the code above the cursor. They ignore definitions elsewhere in the same repository, which matter a great deal in modular projects. Without that information, models invent class methods or call functions with wrong arguments, especially in private, still-developing codebases that never appeared in pre-training data.

The authors propose CoCoMIC, which pairs a retrieval tool with a modelling framework. The tool, CCFinder, uses static analysis to turn a Python project into a directed, multi-relational context graph of entities and their links. It then uses the incomplete file's import statements to pull nearby graph neighbours as cross-file context. CoCoMIC, built on CodeGen, first compresses this retrieved context into compact representations. While generating, it attends to these representations together with the full in-file tokens, which avoids simply concatenating everything into a limited input window.

On a completion benchmark built from PyPI projects, the method clearly improves over fine-tuned baselines. This shows that repository-level context is a valuable and underused signal for code completion.

Key points

  • CCFinder's graph covers four entity types: file, class, function and global variable. These are connected by intra-file (hierarchy) and inter-file (import dependency) relations, and each entity has a unique "locale" path name.
  • Retrieval starts from entities referenced by imports and collects neighbours up to 2 hops away using depth-first search. This yields 27.07% more relevant context than the in-file context alone.
  • With CCFinder context, CoCoMIC improves its CodeGen backbone by 33.94% in exact match and 28.69% in identifier match, both relative.
  • The evaluation data is a Python code completion set built from the Python Package Index (PyPI). The design is meant to extend to other languages.
  • Code is planned for release at github.com/amazon-science/cocomic, and the paper includes ablations on how to integrate cross-file context.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksInformation Systems · 1,437 worksSoftware · 92 worksComputer Science Applications · 102 works

🧭 Topics 1Research subjects and languages the work is about.Code & programming languages · 129 works

Cite

Cite as Yangruibo Ding, Zijian Wang, Wasi Ahmad, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth and Bing Xiang. 2024. CoCoMIC: Code Completion by Jointly Modeling In-file and Cross-file Context. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ding2024cocomic,
  title = {{CoCoMIC: Code Completion by Jointly Modeling In-file and Cross-file Context}},
  author = {Yangruibo Ding, Zijian Wang, Wasi Ahmad, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth and Bing Xiang},
  booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
  year = {2024},
  doi = {10.63317/55z6pyuui3ij},
  url = {https://doi.org/10.63317/55z6pyuui3ij}
}

References

63 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ibrahim Abdelaziz, Julian Dolby, James P Mc-Cusker, and Kavitha Srinivas. 2021. A toolkit for generating code knowledge graphs. The Eleventh International Conference on Knowledge Capture (K-CAP). find
  2. Ibrahim Abdelaziz, Julian Dolby, Jamie McCusker, and Kavitha Srinivas. 2022. Can machines read coding manuals yet? – a benchmark for building better language models for code understanding. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2022). find
  3. Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Unified pre-training for program understanding and generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2655–2668, O…
  4. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. ArXiv preprint, abs/2108.07732.
  5. Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2023. Grounded copilot: How programmers interact with code-generating models. Proc. ACM Program. Lang., 7(OOPSLA1). find
  6. Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021a. Gpt-neo: Large scale autoregressive language modeling with mesh-tensorflow. If you use this software, please cite it using these metadata, 58. find
  7. Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021b. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow. find
  8. Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022. GPT-NeoX-20B: An open-source autor…
  9. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  10. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. ArXiv preprint, abs/2107.03374.
  11. Davor Cubranic and Gail C Murphy. 2003. Hipikat: Recommending pertinent software development artifacts. In 25th International Conference on Software Engineering, 2003. Proceedings., pages 408–418. IEEE. find
  12. Hantian Ding, Jinrui Yang, Yuqian Deng, Hongming Zhang, and Dan Roth. 2022a. Towards open-domain topic classification. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: System Demonstrations, pages 90–98… find
  13. Yangruibo Ding, Luca Buratti, Saurabh Pujar, Alessandro Morari, Baishakhi Ray, and Saikat Chakraborty. 2022b. Towards learning (dis)-similarity of source code from program contrasts. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p… find
  14. Yangruibo Ding, Luca Buratti, Saurabh Pujar, Alessandro Morari, Baishakhi Ray, and Saikat Chakraborty. 2022c. Towards learning (dis)-similarity of source code from program contrasts. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p… find
  15. Yangruibo Ding, Zijian Wang, Wasi Uddin Ahmad, Hantian Ding, Ming Tan, Nihal Jain, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, and Bing Xiang. 2023. Crosscodeeval: A diverse and multilingual benchmark for cross-file code completion. find
  16. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A pre-trained model for programming and natural languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1536–1547,…
  17. Jeanne Ferrante, Karl J. Ottenstein, and Joe D. Warren. 1987. The program dependence graph and its use in optimization. ACM Trans. Program. Lang. Syst., 9(3):319–349. find
  18. Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. 2023. Incoder: A generative model for code infilling and synthesis. In The Eleventh International Conference on Learning Representations. find
  19. Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. UniXcoder: Unified cross-modal pre-training for code representation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7212–7225, Dublin, Ireland. Associa…
  20. Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Jian Yin, Daxin Jiang, et al. 2021. Graphcodebert: Pre-training code representations with data flow. In International Conference on Learning Representations.
  21. Sakib Haque, Alexander LeClair, Lingfei Wu, and Collin McMillan. 2020. Improved automatic summarization of subroutines via attention to file context. In Proceedings of the 17th International Conference on Mining Software Repositories, pages 300–310. find
  22. Scott Henninger. 1991. Retrieving software objects in an example-based programming environment. In Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, pages 251–260. find
  23. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022. Survey of hallucination in natural language generation. ACM Computing Surveys.
  24. Rosco Hill and Joe Rideout. 2004. Automatic method completion. In Proceedings. 19th International Conference on Automated Software Engineering, 2004., pages 228–235. IEEE. find
  25. Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. Open-Review.net. find
  26. Abram Hindle, Earl T. Barr, Zhendong Su, Mark Gabel, and Premkumar Devanbu. 2012. On the naturalness of software. In Proceedings of the 34th International Conference on Software Engineering, ICSE ’12, page 837–847. IEEE Press. find
  27. Reid Holmes and Gail C Murphy. 2005. Using structural context to recommend source code examples. In Proceedings of the 27th international conference on Software engineering, pages 117– 125. find
  28. Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. Codesearchnet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436.
  29. Donald E. Knuth. 1984. Literate programming. Comput. J., 27(2):97–111. find
  30. Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700.
  31. Agathe Lherondelle, Yash Satsangi, Fran Silavong, Shaltiel Eloul, and Sean Moran. 2022. Topical: Learning repository embeddings from source code using attention. ArXiv preprint, abs/2208.09495. — [arXiv:2208.09495](https://arxiv.org/abs/2208.09495) find
  32. Importlab. 2018. importlab: A library for python that automatically infers dependencies and calculates a dependency graph. https://github.com/ google/importlab. ↗
  33. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode. ArXiv preprint, abs/2203.07814.
  34. Katsuro Inoue, Reishi Yokomori, Hikaru Fujiwara, Tetsuo Yamamoto, Makoto Matsushita, and Shinji Kusumoto. 2003. Component rank: Relative significance rank for software component search. In 25th International Conference on Software Engineering, 2003. Proceedings., pages 14–24. IEEE. find
  35. Yuxian Meng, Shi Zong, Xiaoya Li, Xiaofei Sun, Tianwei Zhang, Fei Wu, and Jiwei Li. 2022. GNN-LM: Language modeling based on global contexts via GNN. In International Conference on Learning Representations. find
  36. Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2018. Mapping language to code in programmatic context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1643– 1652, Brussels, Belgium. Association for Computational Linguistics. find
  37. Amir Michail. 2001. Codeweb: Data mining library reuse patterns. In Proceedings of the 23rd International Conference on Software Engineering. find
  38. Paul Jaccard. 1912. The distribution of the flora in the alpine zone. 1. New phytologist, 11(2):37–50. find
  39. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. Codegen: An open large language model for code with multi-turn program synthesis. In The Eleventh International Conference on Learning Representations. find
  40. Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. 2019. What does BERT learn about the structure of language? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3651–3657, Florence, Italy. Association for Computational Linguistics.
  41. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, page 311–318, USA. Association for Computational Linguistics.
  42. D. L. Parnas. 1972. On the criteria to be used in decomposing systems into modules. Commun. ACM, 15(12):1053–1058. find
  43. D.L. Parnas, P.C. Clements, and D.M. Weiss. 1985. The modular structure of complex systems. IEEE Transactions on Software Engineering, SE-11(3):259–266. find
  44. Pydeps. 2021. Python module dependency graphs. https://github.com/thebjorn/pydeps. ↗
  45. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI preprint. find
  46. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333–389. find
  47. Mary Beth Rosson and John M Carroll. 1996. The reuse of uses in smalltalk programming. ACM Transactions on Computer-Human Interaction (TOCHI), 3(3):219–253. find
  48. Disha Shrivastava, Hugo Larochelle, and Daniel Tarlow. 2023. Repository-level prompt generation for large language models of code. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 31693– 31715. PMLR. find
  49. Kevin J. Sullivan, William G. Griswold, Yuan-fang Cai, and Ben Hallen. 2001. The structure and value of modularity in software design. ESEC/FSE-9, page 99–108, New York, NY, USA. Association for Computing Machinery. find
  50. Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, and Denys Poshyvanyk. 2019. An empirical study on learning bug-fixing patches in the wild via neural machine translation. ACM Transactions on Software Engineering and Methodology (TOSEM), 28(4):1–29. find
  51. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. find
  52. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/ kingoflolz/mesh-transformer-jax. ↗
  53. Yanlin Wang, Ensheng Shi, Lun Du, Xiaodi Yang, Yuxuan Hu, Shi Han, Hongyu Zhang, and Dongmei Zhang. 2021a. Cocosum: Contextual code summarization with multi-relational graph neural network. ArXiv preprint, abs/2107.01933. — [arXiv:2107.01933](https://arxiv.org/abs/2107.01933) find
  54. Yue Wang, Weishi Wang, Shafiq Joty, and Steven C.H. Hoi. 2021b. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8696–8708, Online and Punta Cana… find
  55. Wikipedia. 2022. Modular programming. https://en.wikipedia.org/wiki/ Modular_programming. ↗
  56. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… — Transformers: State-of-the-Art Natural Language Processing
  57. Frank F Xu, Uri Alon, Graham Neubig, and Vincent Josua Hellendoorn. 2022. A systematic evaluation of large language models of code. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming, pages 1–10. find
  58. Fabian Yamaguchi, Nico Golde, Daniel Arp, and Konrad Rieck. 2014. Modeling and discovering vulnerabilities with code property graphs. In 2014 IEEE Symposium on Security and Privacy, pages 590–604. find
  59. Yunwen Ye and Gerhard Fischer. 2002. Supporting reuse by delivering task-relevant and personalized information. In Proceedings of the 24th international conference on Software engineering, pages 513–523. find
  60. Yunwen Ye, Gerhard Fischer, and Brent Reeves. 2000. Integrating active information delivery and reuse repository systems. ACM SIGSOFT Software Engineering Notes, 25(6):60–68. find
  61. Fengji Zhang, Bei Chen, Yue Zhang, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. 2023. Repocoder: Repository-level code completion through iterative retrieval and generation. arXiv preprint arXiv:2303.12570. find
  62. Shuyan Zhou, Uri Alon, Frank F. Xu, Zhengbao Jiang, and Graham Neubig. 2023. Docprompting: Generating code by retrieving the docs. In The Eleventh International Conference on Learning Representations. find
  63. Yaqin Zhou, Shangqing Liu, Jingkai Siow, Xiaoning Du, and Yang Liu. 2019. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. In Advances in Neural Information Processing Systems, volume 32, pages 10197–10207. Curran Associates, Inc. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ding-2024-cocomic-code-completion-jointly-modeling/