Elamur Research⌕
☰
home / papers / 2026 / Team Poznan at SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios

Team Poznan at SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios

Paper · 2026 · Dawid Siera, Anatol Kaczmarek, Wiktor Kamzela et al.

Summary

The paper addresses SemEval-2026 Task 13: distinguishing human-written source code from LLM-generated code. The task has three subtasks: binary detection, attributing a snippet to a human or one of ten LLM families, and a four-way split into human, machine, hybrid and adversarial code. The authors argue that surface style cues and purely semantic embeddings are not enough, so they model program structure directly. Snippets are parsed with Tree-sitter into Abstract Syntax Trees and classified with a Graph Neural Network. They then fuse the GNN representation with CodeBERT embeddings, and in one variant also add naming-convention features.

On the official validation sets, the Hybrid GNN-CodeBERT model beats a fine-tuned CodeBERT baseline and a naming-convention MLP baseline on every subtask. The largest gains come in hybrid code detection. The authors did not report hidden test-set results, citing strong data drift in that set, and they note the model does not transfer to unseen programming languages. The work suggests that structural code representations add useful signal for code-authorship detection in education and software security.

Key points

  • Data: Training sets contain 500k samples for Subtask A (C++, Python and Java, algorithmic code), 500k for Subtask B (heavily imbalanced, with 442k human samples) and 900k for Subtask C. Validation sets hold 100k, 100k and 200k samples.
  • Models:
    • A GNN over AST nodes using five node features (type_id, depth, num_children, is_named, length).
    • A Hybrid GNN-CodeBERT that concatenates both embeddings.
    • A Morphology-Augmented variant that adds a 41-dimensional naming-convention vector.
  • Results (validation macro F1, Hybrid GNN vs. CodeBERT):
    • Subtask A: 0.9949 vs. 0.9860.
    • Subtask B: 0.3843 vs. 0.3161; the morphology variant reaches 0.3937.
    • Subtask C: 0.8098 vs. 0.6784.
    • Macro F1 on Subtask B stays low for all models.
  • Efficiency: The standalone GNN has roughly ten times fewer parameters than CodeBERT and is trained from scratch, yet it slightly beats CodeBERT on Subtasks A and C.
  • Code: The implementation is public at github.com/Dawid64/Semeval-machine-generated-code-detection.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Poznań University of TechnologyPoznan · Poland3 works · 1 citation

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 worksHardware and Architecture · 81 works

🧭 Topics 1Research subjects and languages the work is about.Code & programming languages · 129 works

Cite

Cite as Dawid Siera, Anatol Kaczmarek, Wiktor Kamzela, Adam Dobosz and Jakub Dutkiewicz. 2026. Team Poznan at SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.288

@inproceedings{siera2026team,
  title = {{Team Poznan at SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios}},
  author = {Dawid Siera, Anatol Kaczmarek, Wiktor Kamzela, Adam Dobosz and Jakub Dutkiewicz},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.288},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.288}
}

References

21 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 other…
  2. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Codebert: A pre-trained model for programming and natural languages. Preprint, arXiv:2002.08155.
  3. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M. Rush. 2019. Gltr: Statistical detection and visualization of generated text. Preprint, arXiv:1906.04043.
  4. Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022a. Unixcoder: Unified cross-modal pre-training for code representation. Preprint, arXiv:2203.03850.
  5. Dixiao Guo, Anmin Zhou, Liang Liu, Shan Liao, and Lei Zhang. 2022b. A method of source code authorship attribution based on graph neural network. In Proceedings of 2021 Chinese Intelligent Automation Conference, pages 645–657, Singapore. Springer Singapore. find
  6. Thomas Lavergne, Tanguy Urvoy, and François Yvon. 2008. Detecting fake content with relative entropy scoring. find
  7. Phuong T. Nguyen, Juri Di Rocco, Claudio Di Sipio, Riccardo Rubei, Davide Di Ruscio, and Massimiliano Di Penta. 2023. Is this snippet written by chatgpt? an empirical study with a codebert-based classifier. Preprint, arXiv:2307.09381. — [arXiv:2307.09381](https://arxiv.org/abs/2307.09381) find
  8. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — GPT-4 Technical Report
  9. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. SemEval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios. In Proceedings of the 20th International Workshop on Semantic Evaluatio…
  10. Jinyan Su, Terry Zhuo, Di Wang, and Preslav Nakov. 2023. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12395–12412.
  11. Hyunjae Suh, Mahan Tafreshipour, Jiawei Li, Adithya Bhattiprolu, and Iftekhar Ahmed. 2024. An empirical study on automatically detecting ai-generated source code: How far are we? Preprint, arXiv:2411.04299. — [arXiv:2411.04299](https://arxiv.org/abs/2411.04299) find
  12. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6).
  13. Xiaodan Xu, Chao Ni, Xinrong Guo, Shaoxuan Liu, Xiaoya Wang, Kui Liu, and Xiaohu Yang. 2024. Distinguishing llm-generated from human-written code by contrastive learning. Preprint, arXiv:2411.04704. — [arXiv:2411.04704](https://arxiv.org/abs/2411.04704) find
  14. Zhenyu Xu and Victor S Sheng. 2025. Codevision: Detecting llm-generated code using 2d token probability maps and vision models. arXiv preprint arXiv:2501.03288. — [arXiv:2501.03288](https://arxiv.org/abs/2501.03288) find
  15. Tong Ye, Yangkai Du, Tengfei Ma, Lingfei Wu, Xuhong Zhang, Shouling Ji, and Wenhai Wang. 2024. Uncovering llm-generated code: A zero-shot synthetic code detector via code rewriting. Preprint, arXiv:2405.16133. — [arXiv:2405.16133](https://arxiv.org/abs/2405.16133) find
  16. Shinwoo Park, Hyundong Jin, Jeong-won Cha, and Yo-Sub Han. 2025. Detection of llm-paraphrased code and identification of the responsible llm using coding style features. arXiv preprint arXiv:2502.17749. — [arXiv:2502.17749](https://arxiv.org/abs/2502.17749) find
  17. Shushanta Pudasaini, Luis Miralles-Pechuán, David Lillis, and Marisa Llorens Salvador. 2024. Survey on plagiarism detection in large language models: The impact of chatgpt and gemini on academic integrity. Preprint, arXiv:2407.13105. — [arXiv:2407.13105](https://arxiv.org/abs/2407.13105) find
  18. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2023. Exploring the limits of transfer learning with a unified text-to-text transformer. Preprint, arXiv:1910.10683.
  19. Juan Ramos. 2003. Using tf-idf to determine word relevance in document queries. find
  20. SemEval. 2026. SemEval-2026-Task13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios. https://github.com/mbzuai-nlp/ SemEval-2026-Task13. GitHub repository, accessed 2026-03-02. ↗
  21. Yuling Shi, Hongyu Zhang, Chengcheng Wan, and Xiaodong Gu. 2024. Between lines of code: Unraveling the distinct patterns of machine and human programmers. Preprint, arXiv:2401.06461. — [arXiv:2401.06461](https://arxiv.org/abs/2401.06461) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/siera-2026-team-poznan-semeval-2026-task/