Elamur Research⌕
☰
home / papers / 2026 / Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation

Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation

Paper · 2026 · Jiwon Moon, Yerin Hwang, Dongryeol Lee et al.

Summary

LLMs are increasingly used to judge whether generated code is correct without reference solutions or tests. This paper asks whether such judges stay consistent when code keeps the same behavior but changes on the surface. The authors identify six kinds of bias: cues that appeal to authority, comments in which the code claims to be correct, changed variable names, the inverse of the authority cue, comments that misrepresent the task, and complexity that is only apparent. They build a benchmark in five languages (C++, Python, Java, JavaScript, Go). Each language has 200 task descriptions paired with correct and incorrect solutions, and the biases are injected into those solutions. Every tested judge was affected. Some cues pushed verdicts toward "correct": expert-style comments, self-affirming comments, and longer variable names. Others pushed toward "incorrect": misleading task comments and novice-author remarks. A larger model did not guarantee more robustness. Asking judges to write test cases before deciding helped only slightly. This matters because reference-free LLM judging of code can be manipulated by trivial edits. Such edits can inflate scores for wrong code or penalize correct code, so more robust evaluation methods are needed.

Key points

  • The benchmark covers 5 programming languages with 200 tasks each. Correct and incorrect solutions are perturbed with 6 bias types, and the effects are classed as positive bias (favoring a "correct" verdict) or negative bias (favoring "incorrect").
  • GPT-4o's accuracy fell by as much as 26.7 percentage points under biased inputs.
  • Renaming variables to random strings of only two characters already produced consistent positive bias, which grew with name length. Single-character names had the opposite, negative effect.
  • Adding more meaningless dummy functions steadily strengthened positive bias. With 8 dummy functions, accuracy on correct code rose from 79.67 to 89.33, while accuracy on incorrect code dropped from 63.50 to 46.65.
  • Test-case-generation prompting reduced MAD only modestly and resisted negative biases somewhat better. Positive biases stayed, and accuracy on unbiased inputs fell from 71.6% to 66.75%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Seoul National UniversitySeoul · South Korea41 works · 45,671 citations🏢LG AI ResearchSeoul · South Korea7 works · 374 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksInformation Systems · 1,437 worksSoftware · 92 worksComputer Science Applications · 102 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Jiwon Moon, Yerin Hwang, Dongryeol Lee, Taegwan Kang, Yongil Kim and Kyomin Jung. 2026. Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.70

@inproceedings{moon2026don,
  title = {{Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluation}},
  author = {Jiwon Moon, Yerin Hwang, Dongryeol Lee, Taegwan Kang, Yongil Kim and Kyomin Jung},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.70},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.70}
}

References

53 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Pooja Aggarwal, Oishik Chatterjee, Ting Dai, Prateeti Mohapatra, Brent Paulovicks, Brad Blancett, and Arthur De Magalhaes. 2024. Codesift: An llm-based reference-less framework for automatic code validation. In 2024 IEEE 17th International Conference on Cloud Computing (CLOUD), pages 404–410. IEEE. find
  3. Anthropic. 2024. Claude 3.5 sonnet. find
  4. Anna Bavaresco, Raffaella Bernardi, Leonardo Bertolazzi, Desmond Elliott, Raquel Fernández, Albert Gatt, Esam Ghaleb, Mario Giulianelli, Michael Hanna, Alexander Koller, and 1 others. 2024. Llms instead of human judges? a large scale empirical study across 20 nlp evaluation tasks. arXiv preprint arX…
  5. Riccardo Cantini, Alessio Orsino, Massimo Ruggiero, and Domenico Talia. 2025. Benchmarking adversarial robustness to bias elicitation in large language models: Scalable automated assessment with llm-asa-judge. arXiv preprint arXiv:2504.07887. — [arXiv:2504.07887](https://arxiv.org/abs/2504.07887) find
  6. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201.
  7. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2022. Codet: Code generation with generated tests. arXiv preprint arXiv:2207.10397. find
  8. Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. 2024a. Mllm-asa-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark. In Forty-first International Conference on Machine Learning. find
  9. Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. 2024b. Humans or llms as the judge? a study on judgement biases. arXiv preprint arXiv:2402.10669.
  10. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  11. Enrique Dehaerne, Bappaditya Dey, Sandip Halder, Stefan De Gendt, and Wannes Meert. 2022. Code generation using machine learning: A systematic review. Ieee Access, 10:82434–82455. find
  12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
  13. Mikhail Evtikhiev, Egor Bogomolov, Yaroslav Sokolov, and Timofey Bryksin. 2023. Out of the bleu: how should we assess quality of the code generation models? Journal of Systems and Software, 203:111741. find
  14. Google. 2025. Gemini 2.0 flash. find
  15. Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, and 1 others. 2024. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594. find
  16. Johannes C Hofmeister, Janet Siegmund, and Daniel V Holt. 2019. Shorter identifier names take longer to comprehend. Empirical Software Engineering, 24:417–443. find
  17. Yerin Hwang, Yongil Kim, Jahyun Koo, Taegwan Kang, Hyunkyung Bae, and Kyomin Jung. 2025. Llms can be easily confused by instructional distractions. arXiv preprint arXiv:2502.04362. find
  18. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP (1), pages 6769–6781.
  19. Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. 2023. Benchmarking cognitive biases in large language models as evaluators. arXiv preprint arXiv:2309.17012. find
  20. Sumith Kulal, Panupong Pasupat, Kartik Chandra, Mina Lee, Oded Padon, Alex Aiken, and Percy S Liang. 2019. Spoc: Search-based pseudocode to code. Advances in Neural Information Processing Systems, 32. find
  21. Dongryeol Lee, Yerin Hwang, Yongil Kim, Joonsuk Park, and Kyomin Jung. 2024. Are llm-judges robust to expressions of uncertainty? investigating the effect of epistemic markers on llm-based evaluation. arXiv preprint arXiv:2410.20774. find
  22. Kefan Li and Yuan Yuan. 2024. Large language models as test case generators: Performance evaluation and enhancement. arXiv preprint arXiv:2404.13340. — [arXiv:2404.13340](https://arxiv.org/abs/2404.13340) find
  23. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  24. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023a. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36:21558– 21572.
  25. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023b. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634.
  26. Delano Oliveira, Reydne Santos, Fernanda Madeiral, Hidehiko Masuhara, and Fernando Castor. 2023. A systematic literature review on the impact of formatting elements on code legibility. Journal of Systems and Software, 203:111728. find
  27. OpenAI. 2024a. Gpt-4o mini: advancing cost-efficient intelligence. find
  28. OpenAI. 2024b. Hello gpt-4o. find
  29. Shuyin Ouyang, Jie M Zhang, Mark Harman, and Meng Wang. 2025. An empirical study of the non-determinism of chatgpt in code generation. ACM Transactions on Software Engineering and Methodology, 34(2):1–28. find
  30. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  31. Pranjal Kumar. 2024. Large language models (llms): survey, technical frameworks, and future challenges. Artificial Intelligence Review, 57(10):260. find
  32. Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, and 1 others. 2021. Codenet: A large-scale ai for code dataset for learning a diversity of coding tasks. arXiv preprint arXiv:2105.12655. find
  33. Dawn Lawrie, Christopher Morrell, Henry Feild, and David Binkley. 2006. What’s in a name? a study of identifiers. In 14th IEEE international conference on program comprehension (ICPC’06), pages 3–12. IEEE. find
  34. Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. 2020. Codebleu: a method for automatic evaluation of code synthesis. arXiv preprint arXiv:2009.10297. find
  35. LG Research, Soyoung An, Kyunghoon Bae, Eunbi Choi, Kibong Choi, Stanley Jungkyu Choi, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Gerrard Jeong-won Jo, and 1 others. 2024. Exaone 3.5: Series of large language models for real-world use cases. arXiv preprint arXiv:2412.04862. — [arXiv:2412.04862](https://arxiv.org/abs/2412.04862) find
  36. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950.
  37. Lin Shi, Chiyu Ma, Wenhua Liang, Weicheng Ma, and Soroush Vosoughi. 2024. Judging the judges: A systematic investigation of position bias in pairwise comparative assessments by llms. arXiv preprint arXiv:2406.07791. — [arXiv:2406.07791](https://arxiv.org/abs/2406.07791) find
  38. Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. 2024. Judgebench: A benchmark for evaluating llm-based judges. arXiv preprint arXiv:2410.12784. find
  39. Weixi Tong and Tianyi Zhang. 2024. Codejudge: Evaluating code generation with large language models. arXiv preprint arXiv:2410.02184. find
  40. Ngoc Tran, Hieu Tran, Son Nguyen, Hoan Nguyen, and Tien Nguyen. 2019. Does bleu score work for code migration? In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC), pages 165–176. IEEE. find
  41. Jianxun Wang and Yixiang Chen. 2023. A review on code generation with llms: Application and evaluation. In 2023 IEEE International Conference on Medical Artificial Intelligence (MedAI), pages 284– 289. IEEE. find
  42. Ruiqi Wang, Jiyu Guo, Cuiyun Gao, Guodong Fan, Chun Yong Chong, and Xin Xia. 2025. Can llms replace human evaluators? an empirical study of llm-as-a-judge in software engineering. arXiv preprint arXiv:2502.06193. — [arXiv:2502.06193](https://arxiv.org/abs/2502.06193) find
  43. Shengwei Xu, Yuxuan Lu, Grant Schoenebeck, and Yuqing Kong. 2024. Benchmarking llms’ judgments with no gold standard. arXiv preprint arXiv:2411.07127. — [arXiv:2411.07127](https://arxiv.org/abs/2411.07127) find
  44. Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, and 1 others. 2024. Justice or prejudice? quantifying biases in llm-as-ajudge. arXiv preprint arXiv:2410.02736.
  45. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  46. Yuwei Zhao, Ziyang Luo, Yuchen Tian, Hongzhan Lin, Weixiang Yan, Annan Li, and Jing Ma. 2024. Codejudge-eval: Can large language models be good judges in code understanding? arXiv preprint arXiv:2408.10718. find
  47. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595–46623.
  48. Shuyan Zhou, Uri Alon, Sumit Agarwal, and Graham Neubig. 2023. Codebertscore: Evaluating code generation with pretrained models of code. arXiv preprint arXiv:2302.05527. find
  49. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. Judgelm: Fine-tuned large language models are scalable judges. arXiv preprint arXiv:2310.17631.
  50. Terry Yue Zhuo. 2024. Ice-score: Instructing large language models to evaluate code. In Findings of the Association for Computational Linguistics: EACL 2024, pages 2232–2242. find
  51. Yanlin Wang, Tianyue Jiang, Mingwei Liu, Jiachi Chen, and Zibin Zheng. 2024. Beyond functional correctness: Investigating coding style inconsistencies in large language models. arXiv preprint arXiv:2407.00456. — [arXiv:2407.00456](https://arxiv.org/abs/2407.00456) find
  52. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
  53. Minghao Wu and Alham Fikri Aji. 2023. Style over substance: Evaluation biases for large language models. arXiv preprint arXiv:2307.03025.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/moon-2026-don-t-judge-code-its/