Elamur Research⌕
☰
home / papers / 2026 / Pixel Phantoms at SemEval-2026 Task 13: Exploring Classical and Neural Approaches for AI-Generated Code Detection

Pixel Phantoms at SemEval-2026 Task 13: Exploring Classical and Neural Approaches for AI-Generated Code Detection

Paper · 2026 · Jithu Morrison S, Janani Hariharakrishnan, Angel Deborah S et al.

Summary

This paper describes a team's entry for SemEval-2026 Task 13, Subtask A. The task is binary: decide whether a code snippet was written by a human or produced by AI, using the official parquet train, validation and test splits. The authors compare two families of systems. The first is classical TF-IDF classifiers: Logistic Regression on character n-grams and LinearSVC on code-aware word n-grams. The second is fine-tuned code transformers, CodeBERT and GraphCodeBERT, trained with Hugging Face on a Colab T4 GPU. GraphCodeBERT, whose pre-training uses data flow graphs, achieved the best leaderboard macro F1 (0.36866).

CodeBERT got steadily worse the longer it was fine-tuned. Its earliest checkpoint scored best, which the authors attribute to overfitting to surface artifacts. A cheap character TF-IDF model nearly matched the best CodeBERT run. The work is an empirical comparison, not a new architecture. Its practical lessons are that checkpoint selection matters and that lexical baselines remain strong. Overall scores still trail the top leaderboard systems.

Key points

  • Leaderboard results (macro F1):

    • GraphCodeBERT: 0.36866
    • CodeBERT, early checkpoint: 0.32976
    • Logistic Regression with char TF-IDF (3–5-grams, 5,000 features): 0.32854
    • LinearSVC with word TF-IDF: about 0.28–0.29

  • CodeBERT degradation: macro F1 declined from 0.32976 to 0.28188 (checkpoint-13000) and reached 0.27664 by the final checkpoint, an overall loss of 0.05312. The paper labels the late checkpoint inconsistently, as 63,000 in some places and 65,000 in others.

  • Transformer setup: both models used microsoft/codebert-base and microsoft/graphcodebert-base settings of 256 tokens, batch size 8, lr 2×10⁻⁵, 3 epochs, fp16 and gradient checkpointing.

  • GraphCodeBERT margins: it beat the best CodeBERT run by 0.03890 and the best TF-IDF baseline by 0.04012. The gains were clearest on long, comment-heavy snippets with many variable interactions.

  • Recurring errors:

    • very short snippets (under 5 lines);
    • heavily commented code, which the TF-IDF model tended to label as human;
    • boilerplate such as getters/setters, which caused false positives.

    TF-IDF+transformer ensembles were left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksInformation Systems · 1,730 works

Cite

Cite as Jithu Morrison S, Janani Hariharakrishnan, Angel Deborah S and Rajalakshmi S. 2026. Pixel Phantoms at SemEval-2026 Task 13: Exploring Classical and Neural Approaches for AI-Generated Code Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.256

@inproceedings{s2026pixel,
  title = {{Pixel Phantoms at SemEval-2026 Task 13: Exploring Classical and Neural Approaches for AI-Generated Code Detection}},
  author = {Jithu Morrison S, Janani Hariharakrishnan, Angel Deborah S and Rajalakshmi S},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.256},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.256}
}

References

16 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Unified pre-training for program understanding and generation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
  2. Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. 2019. Code2vec: Learning distributed representations of code. In Proceedings of the ACM on Programming Languages, volume 3, pages 40:1–40:29. find
  3. Aylin Caliskan-Islam, Richard Harang, Andrew Liu, Arvind Narayanan, Clare Voss, Fabian Yamaguchi, and Rachel Greenstadt. 2015. De-anonymizing programmers via code stylometry. In 24th USENIX Security Symposium, pages 255–270. find
  4. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shen, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Codebert: A pre-trained model for programming and natural languages. In Findings of the Association for Computational Linguistics: EMNLP 2020.
  5. Georgia Frantzeskou, Stephen G. MacDonell, Efstathios Stamatatos, and Stefanos Gritzalis. 2006. Source code author identification based on n-gram author profiles. Artificial Intelligence Applications and Innovations, pages 508–515. find
  6. Daniel Fried, Abhijit Jain, Iz Beltagy, Dan Orlanski, Daniel Khashabi, Wen-tau Yih, Andreas Stolcke, Huaiyu Zhao, and Noah A. Smith. 2022. Incoder: A generative model for code infilling and synthesis. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. find
  7. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M. Rush. 2019. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116.
  8. Daya Guo, Shuai Lu, Shuo Ren, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, and Nan Duan. 2022. Unixcoder: Unified cross-modal pre-training for code representation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language…
  9. Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shuai Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. Graphcodebert: Pre-training code representations with data flo…
  10. Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Binoculars: Zero-shot way to distinguish between human-written and machine-generated text. arXiv, arXiv:2401.12070.
  11. Sheng Lu, Daya Guo, Shuo Ren, Nan Duan, Neel Sundaresan, Vishal Udandarao, Yuning Ma, Abderrahmane Zghidi, Param Raja, and Anjana Krishna. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv, arXiv:2102.04664. find
  12. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. Proceedings of the 40th International Conference on Machine Learning.
  13. Daniil Orel, Dilshod Azizov, and Preslav Nakov. 2025a. Codet-m4: Detecting machine-generated code in multi-lingual, multi-generator and multi-domain settings. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10570–10593, Vienna, Austria. Association for Computational Lin… find
  14. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. Semeval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios. Official SemEval-2026 Task 13 overview page.
  15. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. 2025b. Droid: A resource suite for ai-generated code detection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31251–31277. find
  16. Yue Wang, Weishi Wang, Shafiq Joty, and Steven C. H. Hoi. 2021. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/s-2026-pixel-phantoms-semeval-2026-task-2026-semeval-1-256/