Elamur Research⌕
☰
home / papers / 2026 / LogicEval: A Systematic Framework for Evaluating Automated Repair Techniques for Logical Vulnerabilities in Real-World Software

LogicEval: A Systematic Framework for Evaluating Automated Repair Techniques for Logical Vulnerabilities in Real-World Software

Paper · 2026 · Syed Md Mukit Rashid, Abdullah Al Ishtiaq, Kai Tu et al.

Summary

Most automated vulnerability repair (AVR) research targets memory-safety bugs. Logical vulnerabilities are flaws in program logic, such as missing checks, misplaced validation or skipped state transitions. They rarely crash a program, follow no reusable fix templates, and give sanitizers and runtime logs little to detect.

The authors build LogicDS, a manually curated dataset of real-world logical vulnerabilities with security impact. Each entry contains vulnerable and fixed code, a description and, where available, specifications and developer repair rationales. They also build LogicEval, an end-to-end framework that:

  • feeds localized vulnerable code plus optional context to repair tools,
  • requires structured patch output so patches can be grafted into the source,
  • checks patches by compilation and testing,
  • scores patch quality with automated reasoning metrics (similarity to the ground-truth fix, LLM judges) plus manual review.

The framework compares non-learning, learning-based and off-the-shelf LLM repair approaches. It also varies prompting strategy, repair complexity and auxiliary information. Failures stem mainly from prompt sensitivity, lost code context and trouble placing the patch correctly. This gives the first systematic basis for extending AVR beyond memory corruption.

Key points

  • The paper calls LogicDS the first dataset of logical vulnerabilities. The abstract gives 122 vulnerabilities, while the introduction states 61, so the exact count should be checked against the paper.
  • A motivating example is CVE-2019-1543 in OpenSSL. Its ChaCha20-Poly1305 code accepted nonce lengths up to 16 bytes, although RFC 7539 requires 12 bytes, which risks nonce reuse.
  • The evaluation assumes perfect vulnerability detection and localization, so that patch generation is assessed in isolation.
  • Every plausible patch (one that compiled and passed tests) scored above the 80th percentile in cosine similarity to the ground-truth fix.
  • Qwen3, Llama and OpenAI models serve as judges, and no model judges its own patches, to avoid self-evaluation bias. Qwen gives systematically lower scores, but the trends across prompts stay the same.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Pennsylvania State UniversityState College · United States56 works · 5,360 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksInformation Systems · 1,437 worksSoftware · 92 works

Cite

Cite as Syed Md Mukit Rashid, Abdullah Al Ishtiaq, Kai Tu, Yilu Dong, Tianwei Wu, Ali Ranjbar, Tianchang Yang, Najrin Sultana, Shagufta Mehnaz and Syed Rafiul Hussain. 2026. LogicEval: A Systematic Framework for Evaluating Automated Repair Techniques for Logical Vulnerabilities in Real-World Software. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.2136

@inproceedings{rashid2026logiceval,
  title = {{LogicEval: A Systematic Framework for Evaluating Automated Repair Techniques for Logical Vulnerabilities in Real-World Software}},
  author = {Syed Md Mukit Rashid, Abdullah {Al Ishtiaq}, Kai Tu, Yilu Dong, Tianwei Wu, Ali Ranjbar, Tianchang Yang, Najrin Sultana, Shagufta Mehnaz and Syed Rafiul Hussain},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2136},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2136}
}

References

77 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. https://github.com/rjust/defects4j/blob/ master/framework/projects/Time/patches/3. src.patch. [link]. Defects4J Time Patch 3. https://github.com/ rjust/defects4j/blob/master/framework/ projects/Time/patches/3.src.patch. Fix for cve-2014-0224. https://github.com/openssl/openssl/commit/ bc8923b1ec9c4… ↗
  2. National Vulnerability Database. https://nvd.nist.gov/. RFC 7539. https://datatracker.ietf.org/doc/ html/rfc7539. 2015. CVE-2015-1793. https://nvd.nist.gov/ vuln/detail/CVE-2015-1793. 2017. CVE-2017-3142. https://nvd.nist.gov/ vuln/detail/CVE-2017-3142. 2019a. https://nvd.nist.gov/vuln/detail/ CVE-… ↗
  3. Xi Cheng, Min Zhou, Xiaoyu Song, Ming Gu, and Jiaguang Sun. 2019. Automatic integer error repair by proper-type inference. IEEE Transactions on Dependable and Secure Computing, 18(2):918–935. 2019f. https://github.com/openssl/openssl/blob/ ee22257b1418438ebaf54df98af4e24f494d1809/ crypto/evp/e_chac… ↗
  4. Nariyoshi Chida and Tachio Terauchi. 2022. Repairing dos vulnerability of real-world regexes. In 2022 IEEE Symposium on Security and Privacy (SP), pages 2060–2077. find
  5. IEEE. 2019g. CVE-2019-1543. https://nvd.nist.gov/ vuln/detail/CVE-2019-1543. ↗
  6. Ganesan Deepa, P Santhi Thilagam, Amit Praseed, and Alwyn R Pais. 2018. Detlogic: A black-box approach for detecting logic vulnerabilities in web applications. Journal of Network and Computer Applications, 109:89–109. 2021. Codeql for research. https://securitylab.github.com/tools/codeql/. 2022. CV… ↗
  7. Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2007. On the accuracy of spectrum-based fault localization. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART-MUTATION 2007), pages 89–98. IEEE. find
  8. Hiralal Agrawal and Joseph R Horgan. 1990. Dynamic program slicing. ACM SIGPlan Notices, 25(6):246– 256. find
  9. Guru Bhandari, Amara Naseer, and Leon Moonen. 2021. Cvefixes: automated collection of vulnerabilities and their fixes from open-source software. In Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering, pages 30–39. find
  10. Quang-Cuong Bui, Riccardo Scandariato, and Nicolás E Díaz Ferreyra. 2022. Vul4j: a dataset of reproducible java vulnerabilities geared towards the study of program repair techniques. In Proceedings of the 19th International Conference on Mining Software Repositories, pages 464–468. find
  11. Gelei Deng, Yi Liu, Víctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu, Martin Pinzger, and Stefan Rass. 2024. {PentestGPT}: Evaluating and harnessing large language models for automated penetration testing. In 33rd USENIX Security Symposium (USENIX Security 24), pages 84… find
  12. Yilu Dong, Tianchang Yang, Abdullah Al Ishtiaq, Syed Md Mukit Rashid, Ali Ranjbar, Kai Tu, Tianwei Wu, Md Sultan Mahmud, and Syed Rafiul Hussain. 2025. Corecrisis: Threat-guided and context-aware iterative learning and fuzzing of 5g core networks. In 34th USENIX Security Symposium (USENIX Security 2… find
  13. Jiahao Fan, Yi Li, Shaohua Wang, and Tien N Nguyen. 2020. Ac/c++ code vulnerability dataset with code changes and cve summaries. In Proceedings of the 17th International Conference on Mining Software Repositories, pages 508–512. find
  14. Chongzhou Fang, Ning Miao, Shaurya Srivastav, Jialin Liu, Ruoyu Zhang, Ruijie Fang, Ryan Tsang, Najmeh Nazari, Han Wang, Houman Homayoun, and 1 others. 2024. Large language models for code analysis: Do {LLMs} really do their job? In 33rd USENIX Security Symposium (USENIX Security 24), pages 829–846. find
  15. Viktoria Felmetsger, Ludovico Cavedon, Christopher Kruegel, and Giovanni Vigna. 2010. Toward automated detection of logic vulnerabilities in web applications. In 19th USENIX Security Symposium (USENIX Security 10). find
  16. José Campos, André Riboira, Alexandre Perez, and Rui Abreu. 2012. Gzoltar: an eclipse plug-in for testing and debugging. In Proceedings of the 27th IEEE/ACM international conference on automated software engineering, pages 378–381. find
  17. Xiang Gao, Bo Wang, Gregory J Duck, Ruyi Ji, Yingfei Xiong, and Abhik Roychoudhury. 2021. Beyond tests: Program vulnerability repair via crash constraint extraction. ACM Transactions on Software Engineering and Methodology (TOSEM), 30(2):1– 27. find
  18. Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, and Martin Monperrus. 2019. Sequencer: Sequence-to-sequence learning for end-to-end program repair. IEEE Transactions on Software Engineering, 47(9):1943–1959. find
  19. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  20. Zhen Huang, David Lie, Gang Tan, and Trent Jaeger. 2019. Using safety properties to generate vulnerability patches. In 2019 IEEE symposium on security and privacy (SP), pages 539–554. IEEE. find
  21. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186.
  22. Jiajun Jiang, Yingfei Xiong, Hongyu Zhang, Qing Gao, and Xiangqun Chen. 2018. Shaping program repair space with existing patches and similar code. In Proceedings of the 27th ACM SIGSOFT international symposium on software testing and analysis, pages 298–309. and patch validation feedback. In Proceed… find
  23. Xuan Bach D Le, David Lo, and Claire Le Goues. 2016. History driven program repair. In 2016 IEEE 23rd international conference on software analysis, evolution, and reengineering (SANER), volume 1, pages 213–224. IEEE. find
  24. Thanh Le-Cong, Duc-Minh Luong, Xuan Bach D Le, David Lo, Nhat-Hoa Tran, Bui Quang-Huy, and Quyet-Thang Huynh. 2023. Invalidator: Automated patch correctness assessment via semantic and syntactic reasoning. IEEE Transactions on Software Engineering, 49(6):3411–3429. find
  25. Nan Jiang, Thibaud Lutellier, Yiling Lou, Lin Tan, Dan Goldwasser, and Xiangyu Zhang. 2023. Knod: Domain knowledge distilled tree decoder for automated program repair. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), pages 1251–1263. IEEE. find
  26. Claire Le Goues, Michael Dewey-Vogt, Stephanie Forrest, and Westley Weimer. 2012. A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each. In 2012 34th international conference on software engineering (ICSE), pages 3–13. IEEE. find
  27. Nan Jiang, Thibaud Lutellier, and Lin Tan. 2021. Cure: Code-aware neural machine translation for automatic program repair. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 1161–1173. IEEE. find
  28. Frank Li and Vern Paxson. 2017. A large-scale empirical study of security patches. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 2201–2215. find
  29. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770.
  30. Hanlei Jin, Yang Zhang, Dan Meng, Jun Wang, and Jinghua Tan. 2024. A comprehensive survey on process-oriented automatic text summarization with exploration of llm-based methods. arXiv preprint arXiv:2403.02901. — [arXiv:2403.02901](https://arxiv.org/abs/2403.02901) find
  31. Matthew Jin, Syed Shahriar, Michele Tufano, Xin Shi, Shuai Lu, Neel Sundaresan, and Alexey Svyatkovskiy. 2023. Inferfix: End-to-end program repair with llms. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages… find
  32. James A Jones and Mary Jean Harrold. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, pages 273–282. find
  33. René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4j: A database of existing faults to enable controlled testing studies for java programs. In Proceedings of the 2014 international symposium on software testing and analysis, pages 437–440. find
  34. Ummay Kulsum, Haotian Zhu, Bowen Xu, and Marcelo d’Amorim. 2024. A case study of llm for automated vulnerability repair: Assessing impact of reasoning Yi Li, Shaohua Wang, and Tien N Nguyen. 2020. Dlfix: Context-based code transformation learning for automated program repair. In Proceedings of the A… find
  35. Ying Li, Gang Wang, Yuan Tian, and 1 others. 2025. Sok: Towards effective automated vulnerability repair. arXiv preprint arXiv:2501.18820. — [arXiv:2501.18820](https://arxiv.org/abs/2501.18820) find
  36. Yue Li, Xiao Li, Hao Wu, Yue Zhang, Xiuzhen Cheng, Sheng Zhong, and Fengyuan Xu. 2024. Attention is all you need for llm-based code vulnerability localization. arXiv preprint arXiv:2410.15288. — [arXiv:2410.15288](https://arxiv.org/abs/2410.15288) find
  37. Derrick Lin, James Koppel, Angela Chen, and Armando Solar-Lezama. 2017. Quixbugs: A multi-lingual program repair benchmark set based on the quixey challenge. In Proceedings Companion of the 2017 ACM SIGPLAN international conference on systems, programming, languages, and applications: software for h… find
  38. Zhiqiang Lin, Xuxian Jiang, Dongyan Xu, Bing Mao, and Li Xie. 2007. Autopag: towards automated software patch generation with source code root cause identification and repair. In Proceedings of the 2nd ACM symposium on Information, computer and communications security, pages 329–340. find
  39. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2024a. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36.
  40. Peiyu Liu, Junming Liu, Lirong Fu, Kangjie Lu, Yifan Xia, Xuhong Zhang, Wenzhi Chen, Haiqin Weng, Shouling Ji, and Wenhai Wang. 2024b. Exploring {ChatGPT’s} capabilities on vulnerability management. In 33rd USENIX Security Symposium (USENIX Security 24), pages 811–828. find
  41. Yu Liu, Sergey Mechtaev, Pavle Subotić, and Abhik Roychoudhury. 2023. Program repair guided by datalog-defined static analysis. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 1216–1228. find
  42. Thibaud Lutellier, Hung Viet Pham, Lawrence Pang, Yitong Li, Moshi Wei, and Lin Tan. 2020. Coconut: combining context-aware neural translation models using ensemble for program repair. In Proceedings of the 29th ACM SIGSOFT international symposium on software testing and analysis, pages 101–114. find
  43. Alexandru Marginean, Johannes Bader, Satish Chandra, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, and Andrew Scott. 2019. Sapfix: Automated end-to-end repair at scale. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 269–278… find
  44. Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. 2016. Angelix: Scalable multiline program patch synthesis via symbolic analysis. In Proceedings of the 38th international conference on software engineering, pages 691–701. find
  45. Xiang Mei, Pulkit Singh Singaria, Jordi Del Castillo, Haoran Xi, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, Hammond Pearce, Brendan Dolan-Gavitt, and 1 others. 2024. Arvo: Atlas of reproducible vulnerabilities for open source software. arXiv preprint arXiv:2408.02153. — [arXiv:2408.02153](https://arxiv.org/abs/2408.02153) find
  46. Ziyi Ni, Yifan Li, Ning Yang, Dou Shen, Pin Lyu, and Daxiang Dong. 2025. Tree-of-code: A self-growing tree framework for end-to-end code generation and execution in complex tasks. In Findings of the Association for Computational Linguistics: ACL 2025, pages 9804–9819. find
  47. Georgios Nikitopoulos, Konstantina Dritsa, Panos Louridas, and Dimitris Mitropoulos. 2021. Crossvul: a cross-language vulnerability dataset with commit data. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineer… find
  48. Gene Novark, Emery D Berger, and Benjamin G Zorn. 2007. Exterminator: automatically correcting memory errors with high probability. In Proceedings of the 28th ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 1–11. find
  49. Hakjoo Oh. 2018. Memfix: Static analysis-based repair of memory deallocation errors for c. In FSE 2018: ACM SIGSOFT Symposium on the Foundations of Software Engineering. ACM. find
  50. OpenAI. 2025. Openai o3 and o4-mini system card. find
  51. Arjun Panickssery, Samuel R Bowman, and Shi Feng. 2024. Llm evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems, 37:68772–68802. find
  52. Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2025. Asleep at the keyboard? assessing the security of github copilot’s code contributions. Communications of the ACM, 68(2):96–105. find
  53. Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2023. Examining zero-shot vulnerability repair with large language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2339–2356. IEEE. find
  54. Ali Ranjbar, Tianchang Yang, Kai Tu, Saaman Khalilol-lahi, and Syed Rafiul Hussain. 2025. Stateful Analysis and Fuzzing of Commercial Baseband Firmware . In 2025 IEEE Symposium on Security and Privacy (SP), pages 1120–1139, Los Alamitos, CA, USA. IEEE Computer Society. find
  55. Kuniaki Saito, Kihyuk Sohn, Chen-Yu Lee, and Yoshitaka Ushiku. 2024. Unsupervised llm adaptation for question answering. arXiv preprint arXiv:2402.12170. — [arXiv:2402.12170](https://arxiv.org/abs/2402.12170) find
  56. Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt. 2023. Lost at c: A user study on the security implications of large language model code assistants. In 32nd USENIX Security Symposium (USENIX Security 23), pages 2205–2222. find
  57. Ridwan Shariffdeen, Yannic Noller, Lars Grunske, and Abhik Roychoudhury. 2021. Concolic program repair. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pages 390–405. find
  58. Alex Shaw, Dusten Doggett, and Munawar Hafiz. 2014. Automatically fixing c buffer overflows using program transformations. In 2014 44th Annual IEEE/I-FIP International Conference on Dependable Systems and Networks, pages 124–135. IEEE. find
  59. Dokyung Song, Julian Lettner, Prabhu Rajasekaran, Yeoul Na, Stijn Volckaert, Per Larsen, and Michael Franz. 2019. Sok: Sanitizing for security. In 2019 IEEE Symposium on Security and Privacy (SP), pages 1275–1295. IEEE. find
  60. Shin Hwei Tan, Hiroaki Yoshida, Mukul R Prasad, and Abhik Roychoudhury. 2016. Anti-patterns in search-based program repair. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 727– 738. find
  61. Haoye Tian, Kui Liu, Yinghua Li, Abdoul Kader Kaboré, Anil Koyuncu, Andrew Habib, Li Li, Junhao Wen, Jacques Klein, and Tegawendé F Bissyandé. 2023. The best of both worlds: Combining learned embeddings with engineered features for accurate prediction of correct patches. ACM Transactions on Software… find
  62. Haoye Tian, Xunzhu Tang, Andrew Habib, Shang-wen Wang, Kui Liu, Xin Xia, Jacques Klein, and Tegawendé F Bissyandé. 2022. Is this change the answer to that problem? correlating descriptions of bug and code changes for evaluating patch correctness. In Proceedings of the 37th IEEE/ACM International Con… find
  63. Kai Tu, Abdullah Al Ishtiaq, Syed Md Mukit Rashid, Yilu Dong, Weixuan Wang, Tianwei Wu, and Syed Rafiul Hussain. 2024. Logic gone astray: A security analysis framework for the control plane protocols of 5g basebands. In 33rd USENIX Security Symposium (USENIX Security 24), pages 3063–3080. find
  64. Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. 2023. Can large language models identify and reason about security vulnerabilities? not yet. arXiv preprint arXiv:2312.12575. — [arXiv:2312.12575](https://arxiv.org/abs/2312.12575) find
  65. Ashok Urlana, Pruthwik Mishra, Tathagato Roy, and Rahul Mishra. 2024. Controllable text summarization: unraveling challenges, approaches, and prospects-a survey. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1603– 1623. find
  66. Rijnard van Tonder and Claire Le Goues. 2018. Static automated program repair for heap properties. In Proceedings of the 40th International Conference on Software Engineering, pages 151–162. find
  67. Hanbin Wang, Zhenghao Liu, Shuo Wang, Ganqu Cui, Ning Ding, Zhiyuan Liu, and Ge Yu. 2024. Intervenor: Prompting the coding ability of large language models with the interactive chain of repair. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2081–2107. find
  68. Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. 2024. Self-preference bias in llm-as-a-judge. arXiv preprint arXiv:2410.21819.
  69. Qi Xin and Steven P Reiss. 2017. Leveraging syntax-related code for automated program repair. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 660–670. IEEE. find
  70. Yunlong Xing, Shu Wang, Shiyu Sun, Xu He, Kun Sun, and Qi Li. 2024. What {IF} is not enough? fixing null pointer dereference with contextual check. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1367–1382. find
  71. Zhengzi Xu, Yulong Zhang, Longri Zheng, Liangzhao Xia, Chenfu Bao, Zhi Wang, and Yang Liu. 2020. Automatic hot patch generation for android kernels. In 29th USENIX Security Symposium (USENIX Security 20), pages 2397–2414. find
  72. Jifeng Xuan, Matias Martinez, Favio Demarco, Maxime Clement, Sebastian Lamelas Marcote, Thomas Durieux, Daniel Le Berre, and Martin Monperrus. 2016. Nopol: Automatic repair of conditional statement bugs in java programs. IEEE Transactions on Software Engineering, 43(1):34–55. find
  73. Peng Yixing, Quan Wang, Licheng Zhang, Yi Liu, and Zhendong Mao. 2024. Chain-of-question: A progressive question decomposition approach for complex knowledge base question answering. In Findings of the Association for Computational Linguistics ACL 2024, pages 4763–4776. find
  74. Jinhong Yu, Yi Chen, Di Tang, Xiaozhong Liu, XiaoFeng Wang, Chen Wu, and Haixu Tang. 2024. Llm-enhanced software patch localization. arXiv preprint arXiv:2409.06816. — [arXiv:2409.06816](https://arxiv.org/abs/2409.06816) find
  75. Ying Zhang, Ya Xiao, Md Mahir Asef Kabir, Danfeng Yao, and Na Meng. 2022. Example-based vulnerability detection and repair in java code. In Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension, pages 190–201. find
  76. Xin Zhou, Bowen Xu, Kisub Kim, DongGyun Han, Hung Huu Nguyen, Thanh Le-Cong, Junda He, Bach Le, and David Lo. 2024. Leveraging large language model for automatic patch correctness assessment. IEEE Transactions on Software Engineering, 50(11):2865–2883. find
  77. Qihao Zhu, Zeyu Sun, Yuan-an Xiao, Wenjie Zhang, Kang Yuan, Yingfei Xiong, and Lu Zhang. 2021. A syntax-guided edit decoder for neural program repair. In Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pa… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rashid-2026-logiceval-systematic-framework-evaluating-automated/