Elamur Research⌕
☰
home / papers / 2026 / Let the Comments Speak: A Multi-Agent Framework based on Large Language Model for Comment-Guided Code Refactoring

Let the Comments Speak: A Multi-Agent Framework based on Large Language Model for Comment-Guided Code Refactoring

Paper · 2026 · Zixuan Wang, Wutong Yu and Deyu Zhou

Summary

LLM-based refactoring tools tend to treat code mostly as syntax and ignore the intent that comments convey. Drawing on Fowler's view that a spot needing a comment is a cue to extract a method, the authors use comments as semantic guidance for refactoring. Because most real code is poorly commented, they first generate comments rather than rely on existing ones. Their system, MACOR, is a multi-agent pipeline with three stages:

  • Commenter and Judger: a Commenter samples several comment-augmented versions of the input, and a Judger picks the most consistent one.
  • Refactorer: the chosen version is used to retrieve similar expert refactoring examples, which guide the Refactorer.
  • Test Execution Module: the output is checked with similarity comparison and static analysis, and failures are fed back as logs for further iterations.

MACOR was tested on three benchmarks with three base LLMs. It improved code quality over representative baselines such as Muarf and received higher developer acceptance in human evaluation. The work shows that synthesized comments can make automated refactoring more intent-aware.

Key points

  • The motivation cites prior findings (Huang et al., 2023) that over 84% of codebases lack adequate comments, so existing comments alone cannot drive refactoring.
  • The framework follows a generate-then-refactor design:
    • K independent comment proposals are generated.
    • A Judger chooses among them using structural stability, semantic preservation and cross-proposal consensus.
  • Validation runs a feedback loop capped at p attempts. Refactored code must pass both similarity comparison against the original and static analysis.
  • Evaluation covered three benchmarks (including EMA-RefA, used in the case study) and three LLMs, using both automated metrics and human evaluation. Specific scores fall in a section omitted from the provided text.
  • The authors list three limitations:
    • MACOR does not handle obsolete or incorrect legacy comments.
    • Retrieval examples are not curated.
    • Evaluation is restricted to method-level, single-file refactoring.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksInformation Systems · 1,437 worksSoftware · 92 works

Cite

Cite as Zixuan Wang, Wutong Yu and Deyu Zhou. 2026. Let the Comments Speak: A Multi-Agent Framework based on Large Language Model for Comment-Guided Code Refactoring. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1620

@inproceedings{wang2026let,
  title = {{Let the Comments Speak: A Multi-Agent Framework based on Large Language Model for Comment-Guided Code Refactoring}},
  author = {Zixuan Wang, Wutong Yu and Deyu Zhou},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1620},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1620}
}

References

43 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Maha Alharbi and Mohammad Alshayeb. 2024. A comparative study of automated refactoring tools. IEEE Access, 12:18764–18781. find
  2. Maurício Aniche, Erick Maziero, Rafael Durelli, and Vinicius H. S. Durelli. 2022. The effectiveness of supervised machine learning algorithms in predicting software refactoring. IEEE Transactions on Software Engineering, 48(4):1432–1450. find
  3. Muhammad Ilyas Azeem, Fabio Palomba, Lin Shi, and Qing Wang. 2019. Machine learning techniques for code smell detection: A systematic literature review and meta-analysis. Information and Software Technology, 108:115–138. find
  4. Qingying Chen and Minghui Zhou. 2018. A neural framework for retrieval and summarization of source code. In 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 826–831. find
  5. Vittorio Cortellessa, Daniele Di Pompeo, Vincenzo Sto-ico, and Michele Tucci. 2023. Many-objective optimization of non-functional attributes based on refactoring of software models. Information and Software Technology, 157:107159. find
  6. Marios Fokaefs, Nikolaos Tsantalis, Eleni Stroulia, and Alexander Chatzigeorgiou. 2011. Jdeodorant: identification and application of extract class refactorings. In 2011 33rd International Conference on Software Engineering (ICSE), pages 1037–1039. find
  7. Martin Fowler. 1999. Refactoring: Improving the Design of Existing Code. Addison-Wesley Object Technology Series. Addison-Wesley. find
  8. Esraa Hamouda, Abeer El-Korany, and Soha Makady. 2025. Smell-ml: A machine learning framework for detecting rarely studied code smells. IEEE Access, 13:12966–12980. find
  9. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2024. MetaGPT: Meta programming for a multi-agent collaborative framework. In The T…
  10. Yuan Huang, Hanyang Guo, Xi Ding, Junhuai Shu, Xiangping Chen, Xiapu Luo, Zibin Zheng, and Xiaocong Zhou. 2023. A comparative study on method comment and inline comment. ACM Trans. Softw. Eng. Methodol., 32(5). find
  11. Di Cui, Qiangqiang Wang, Siqi Wang, Jianlei Chi, Jianan Li, Lu Wang, and Qingshan Li. 2023. Rems: Recommending extract method refactoring opportunities via multi-view representation of code property graph. In 2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC), pages 191–202. find
  12. Kaixin Li, Qisheng Hu, James Xu Zhao, Hui Chen, Yuxi Xie, Tiedong Liu, Michael Shieh, and Junxian He. 2024. InstructCoder: Instruction tuning large language models for code editing. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research… find
  13. Kayla DePalma, Izabel Miminoshvili, Chiara Henselder, Kate Moss, and Eman Abdullah AlOmar. 2024. Exploring chatgpt’s code refactoring capabilities: An empirical study. Expert Syst. Appl., 249(PB). find
  14. Yuding Liang and Kenny Zhu. 2018. Automatic generation of text descriptive comments for code blocks. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). find
  15. Malinda Dilhara, Abhiram Bellur, Timofey Bryksin, and Danny Dig. 2024. Unprecedented code change automation: The fusion of llms and transformation by example. Proc. ACM Softw. Eng., 1(FSE). find
  16. Malinda Dilhara, Ameya Ketkar, and Danny Dig. 2021. Understanding software-2.0: A study of machine learning library usage and evolution. ACM Trans. Softw. Eng. Methodol., 30(4). find
  17. Danilo Leandro Lima, Ronnie De Souza Santos, Guilherme Pires Garcia, Sildemir S. Da Silva, Cesar França, and Luiz Fernando Capretz. 2023. Software testing and code refactoring: A survey with practitioners. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 500… find
  18. Sara Fernandes, Ademar Aguiar, and André Restivo. 2023. Liveref: a tool for live refactoring java code. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, ASE ’22, New York, NY, USA. Association for Computing Machinery. find
  19. Bo Liu, Hui Liu, Guangjie Li, Nan Niu, Zimao Xu, Yifan Wang, Yunni Xia, Yuxia Zhang, and Yanjie Jiang. 2023. Deep learning based feature envy detection boosted by real-world examples. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of So… find
  20. Kathrin Figl, Maria Kirchner, Sebastian Baltes, and Michael Felderer. 2025. The influence of code comments on the perceived helpfulness of stack overflow posts. Empirical Softw. Engg., 30(6). find
  21. Hui Liu, Jiahao Jin, Zhifeng Xu, Yanzhen Zou, Yifan Bu, and Lu Zhang. 2021. Deep learning based code smell detection. IEEE Transactions on Software Engineering, 47(9):1811–1837. find
  22. Wenhao Ma, Yaoxiang Yu, Xiaoming Ruan, and Bo Cai. 2023. Pre-trained model based feature envy detection. In 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), pages 430– 440. find
  23. Luana Martins, Valeria Pontillo, Heitor Costa, Filomena Ferrucci, Fabio Palomba, and Ivan Machado. 2024. Test code refactoring unveiled: where and how does it affect test code quality and effectiveness? Empirical Softw. Engg., 30(1). find
  24. Iman Hemati Moghadam and Mel Ó Cinnéide. 2011. Code-imp: a tool for automated search-based refactoring. In Proceedings of the 4th Workshop on Refactoring Tools, WRT ’11, page 41–44, New York, NY, USA. Association for Computing Machinery. find
  25. Naouel Moha, Yann-Gael Gueheneuc, Laurence Duchien, and Anne-Francoise Le Meur. 2010. Decor: A method for the specification and detection of code and design smells. IEEE Transactions on Software Engineering, 36(1):20–36. find
  26. Niklas Muennighoff, Qian Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von Werra, and Shayne Longpre. 2024. Octopack: Instruction tuning code large language models. In The Twelfth International Conference on Learning Representations. find
  27. Henrique Nunes, Tushar Sharma, and Eduardo Figueiredo. 2025. Marv: A manually validated refactoring dataset. In 2025 IEEE/ACM Second International Conference on AI Foundation Models and Software Engineering (Forge), pages 141–145. find
  28. Ali Ouni, Marouane Kessentini, Houari Sahraoui, and Mohamed Salah Hamdi. 2012. Search-based refactoring: Towards semantics preservation. In 2012 28th IEEE International Conference on Software Maintenance (ICSM), pages 347–356. find
  29. Jevgenija Pantiuchina, Bin Lin, Fiorella Zampetti, Massimiliano Di Penta, Michele Lanza, and Gabriele Bavota. 2021. Why do developers reject refactorings in open-source projects? ACM Trans. Softw. Eng. Methodol., 31(2). find
  30. D. T. W. S. Perera, H. T. M. Premathilake, K. P. H. Thathsarani, R. H. T. Nethmini, D. I. De Silva, and H. M. P. P. K. H. Samarasekara. 2023. Analyzing the impact of code commenting on software quality. In 2023 14th International Conference on Computing Communication and Networking Technologies (ICC… find
  31. Yonnel Chen Kuang Piao, Jean Carlors Paul, Leuson Da Silva, Arghavan Moradi Dakhel, Mohammad Hamdaqa, and Foutse Khomh. 2025. Refactoring with llms: Bridging human expertise and machine understanding. Preprint, arXiv:2510.03914. — [arXiv:2510.03914](https://arxiv.org/abs/2510.03914) find
  32. Dorin Pomian, Abhiram Bellur, Malinda Dilhara, Zarina Kurbatova, Egor Bogomolov, Andrey Sokolov, Timofey Bryksin, and Danny Dig. 2024. Em-assist: Safe automated extractmethod refactoring with llms. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engin… find
  33. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
  34. Atsushi Shirafuji, Yusuke Oda, Jun Suzuki, Makoto Morishita, and Yutaka Watanobe. 2023. Refactoring programs using large language models with few-shot examples. In 2023 30th Asia-Pacific Software Engineering Conference (APSEC), pages 151–160. find
  35. Sewen Thy, Andreea Costea, Kiran Gopinathan, and Ilya Sergey. 2023. Adventure of a lifetime: Extract method refactoring for rust. Proc. ACM Program. Lang., 7(OOPSLA2). find
  36. Changjie Wang, Mariano Scazzariello, Anoud Alsh-nakat, Roberto Guanciale, Dejan Kostić, and Marco Chiesa. 2025a. Dissect-and-restore: Ai-based code verification with transient refactoring. Preprint, arXiv:2510.25406. — [arXiv:2510.25406](https://arxiv.org/abs/2510.25406) find
  37. Siqi Wang, Xing Hu, Xin Xia, and Xinyu Wang. 2025b. Actref: Enhancing the understanding of python code refactoring with action-based analysis. arXiv preprint arXiv:2505.06553. — [arXiv:2505.06553](https://arxiv.org/abs/2505.06553) find
  38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, R…
  39. Di Wu, Fangwen Mu, Lin Shi, Zhaoqiang Guo, Kui Liu, Weiguang Zhuang, Yuqi Zhong, and Li Zhang. 2024. ismell: Assembling llms with expert toolsets for code smell detection and refactoring. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE ’24, page 13… find
  40. Sihan Xu, Aishwarya Sivaraman, Siau-Cheng Khoo, and Jing Xu. 2017. Gems: An extract method refactoring recommender. In 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), pages 24–34. find
  41. Yisen Xu. 2025. Muarf: Leveraging multi-agent workflows for automated code refactoring. In 2025 IEEE/ACM 47th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pages 226–227. find
  42. Yang Zhang, Chuyan Ge, Shuai Hong, Ruili Tian, Chunhao Dong, and Jingjing Liu. 2022. Delesmell: Code smell detection based on deep learning and latent semantic analysis. Know.-Based Syst., 255(C). find
  43. Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. Autocoderover: Autonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, page 1592–1604, New York, NY, USA. Association for Computing Machinery. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-let-comments-speak-multi-agent/