Elamur Research⌕
☰
home / papers / 2026 / LATE-IIMAS at Semeval-2026 Task 13: Evaluating GNNs, PLMs, LLMs, and Stylometry for Automatic Code Identification

LATE-IIMAS at Semeval-2026 Task 13: Evaluating GNNs, PLMs, LLMs, and Stylometry for Automatic Code Identification

Paper · 2026 · Andric Valdez, Emmanuel Ancona, Sebastián Bernardino et al.

Summary

The LATE-IIMAS team (UNAM) took part in SemEval-2026 Task 13, which asks whether source code was written by a person or produced by an AI model. The task has three settings: binary human-vs-machine detection (Subtask A), attribution to one of 11 authors (Subtask B), and a four-way split into human, AI, hybrid, and adversarial code (Subtask C). The team built four separate systems:

  • a GNN over per-snippet token co-occurrence graphs, with nodes initialised from CodeBERT embeddings;
  • a CodeBERT model with domain-adversarial training (DANN) that treats programming language as the domain;
  • few-shot prompting of Cohere's Command A;
  • an XGBoost classifier using handcrafted style features such as length, lexical variety, comments, and indentation.

Because Python dominated the data, they undersampled it for Subtask A and used a two-stage cascade for Subtask B. Validation scores were often high, but they dropped sharply on the hidden test set, which contained more varied languages and generators. The work is useful mainly as a frank report of how poorly code-authorship detectors generalise across languages and generators.

Key points

  • Data: each subtask had roughly 500k–900k training snippets covering as many as 8 languages, namely C, C#, C++, Go, Java, JavaScript, PHP and Python, with Python and Java making up the bulk.
  • Subtask A test F1 (macro):
    • LLM few-shot (Command A, temperature 0.3): 0.5641
    • GNN: 0.5381
    • PLM: 0.4219
    • Stylometry: 0.3200
    • CodeBERT baseline: 0.3053
    • The top leaderboard system reached 0.9971.
  • Validation-to-test drop: the gap was large. For example, the DANN model fell from 0.9774 to 0.4219 on Subtask A, and the GNN fell from 0.8675 to 0.1550 on Subtask B.
  • Subtasks B and C (test F1):
    • The cascaded DANN model, trained first on "Human vs LLM" and then on "LLM only", scored 0.3324 on B against a 0.2285 baseline.
    • On C the PLM scored 0.5677 against a 0.4812 baseline.
  • Final rankings: 38th on Subtask A with the GNN, 25th on B and 22nd on C with the PLM. The code is published on GitHub (PLN-disca-iimas/code-detect-Semeval2026).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Universidad Nacional Autónoma de MéxicoMexico City · Mexico8 works · 288 citations

Also on the byline: Facultad de Estudios Superiores Acatlán.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Andric Valdez, Emmanuel Ancona, Sebastián Bernardino, Helena Gomez-Adorno, Fazlourrahman Balouchzahi and Fabian Herrera. 2026. LATE-IIMAS at Semeval-2026 Task 13: Evaluating GNNs, PLMs, LLMs, and Stylometry for Automatic Code Identification. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.339

@inproceedings{valdez2026late,
  title = {{LATE-IIMAS at Semeval-2026 Task 13: Evaluating GNNs, PLMs, LLMs, and Stylometry for Automatic Code Identification}},
  author = {Andric Valdez, Emmanuel Ancona, Sebastián Bernardino, Helena Gomez-Adorno, Fazlourrahman Balouchzahi and Fabian Herrera},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.339},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.339}
}

References

12 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Uri Alon, Meital Zilberstein, Omer Levy, and Eran Yahav. 2019. code2vec: Learning distributed representations of code. In Proceedings of the ACM on Programming Languages, volume 3, pages 1–29. ACM New York, NY, USA. find
  2. Purushotham Endla, Jayendra Gopal Thatipudi, Madhavi Latha Talluri, Pradeep K Joshi, Amita Joshi, and M.Anuradha. 2025. Autoopticode-llm: An autonomous large language model framework for intelligent code generation and optimization. In 2025 6th International Conference on IoT Based Control Networks… find
  3. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Codebert: A pre-trained model for programming and natural languages. In Findings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMN…
  4. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. Journal of machine learning research, 17(59):1–35. find
  5. Daniil Orel, Dilshod Azizov, and Preslav Nakov. 2025a. CoDet-m4: Detecting machine-generated code in multi-lingual, multi-generator and multi-domain settings. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10570–10593, Vienna, Austria. Association for Computational Lin… find
  6. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. SemEval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios. In Proceedings of the 20th International Workshop on Semantic Evaluatio…
  7. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. 2025b. Droid: A resource suite for ai-generated code detection. Preprint, arXiv:2507.10583. — [arXiv:2507.10583](https://arxiv.org/abs/2507.10583) find
  8. Wei Hung Pan, Ming Jie Chok, Jonathan Leong Shan Wong, Yung Xin Shin, Yeong Shian Poon, Zhou Yang, Chun Yong Chong, David Lo, and Mei Kuan Lim. 2024. Assessing ai detectors in identifying ai-generated code: Implications for education. In Proceedings of the 46th international conference on software e… find
  9. Yuling Shi, Hongyu Zhang, Chengcheng Wan, and Xiaodong Gu. 2025. Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers . In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), pages 1628–1639, Los Alamitos, CA, USA. IEEE Computer Society. find
  10. Cohere Team, A Ahmadian, M Ahmed, J Alammar, M Alizadeh, Y Alnumay, S Althammer, A Arkhangorodsky, V Aryabumi, D Aumiller, and 1 others. 2025. Command a: An enterprise-ready large language model. arXiv preprint arXiv:2504.00698. — [arXiv:2504.00698](https://arxiv.org/abs/2504.00698) find
  11. Andric Valdez-Valenzuela, Helena Gómez-Adorno, and Manuel Montes-y Gómez. 2025. Text graph neural networks for detecting AI-generated content. In Proceedings of the 1stWorkshop on GenAI Content Detection (GenAIDetect), pages 134–139, Abu Dhabi, UAE. International Conference on Computational Linguist… find
  12. Lingfei Wu, Yu Chen, Kai Shen, Xiaojie Guo, Hanning Gao, Shucheng Li, Jian Pei, and Bo Long. 2023. Graph neural networks for natural language processing: A survey. Foundations and Trends in Machine Learning, 16(2):119–328. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/valdez-2026-late-iimas-semeval-2026-task/