Elamur Research⌕
☰
home / papers / 2026 / Team Yuvan at SemEval-2026 Task 13: Task-Adaptive Ensemble Strategies for AI-Generated Code Detection

Team Yuvan at SemEval-2026 Task 13: Task-Adaptive Ensemble Strategies for AI-Generated Code Detection

Paper · 2026 · Yuvan Ramesh and Tongtong Wu

Summary

This system paper tackles SemEval-2026 Task 13, which asks whether a code snippet is machine-written and, if so, which model produced it. The task covers eight programming languages and has three subtasks: binary human-vs-AI detection (A), 11-way source attribution with one human class (B), and 4-way generator attribution (C). Each subtask is scored with macro-F1.

The authors use a different combination of components for each subtask. The components are Qwen2.5-Coder-1.5B-Instruct with LoRA, CodeBERT with head–tail truncation, character n-gram TF-IDF with Logistic Regression, and a LightGBM model over tree-sitter AST features.

  • Task A: neural detectors scored near-perfectly on validation but collapsed on test, with their predictions systematically inverted. AST features transferred far more reliably.
  • Task B: the labels are highly imbalanced, and inverse-frequency class weighting kept the model from predicting only the majority class.
  • Task C: weighted probability averaging of heterogeneous models gave the best results.

Final test scores are 0.638 (A), 0.449 (B) and 0.714 (C). The work shows that robust code-provenance detection depends on matching representations and training objectives to each task's distribution shift and imbalance.

Key points

  • Task A: neural predictions were inverted on test, and flipping them raised test F1 (CodeBERT from 0.245 to 0.561, Qwen from 0.269 to 0.587). The final model blends 0.95 of a 41-feature AST LightGBM with 0.05 of the flipped Qwen output. The AST model lost only about 4% from validation to test, versus about 73% for Qwen.
  • Task B: the Human class makes up 88.3% of training data, a 224:1 imbalance. The authors trained an end-to-end 11-way Qwen classifier with weighted cross-entropy on a 300K stratified subset. Class weights ranged from 0.10 to 23.10. This was preferred over a two-stage pipeline, whose early errors propagated.
  • Task C: Qwen, CodeBERT and TF-IDF were combined, with weights chosen by grid search in steps of 0.05. Results were stable across three random seeds (test F1 0.712 ± 0.002).
  • LoRA setup: r=16, α=32, giving 18.5M trainable parameters. Total compute was about 119 A100 GPU-hours, roughly half of it spent on Task B.
  • Baselines: the system beat all implemented baselines on every subtask, with an average macro-F1 of 0.60 versus 0.52 for CodeBERT and 0.40 for TF-IDF+LR.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Monash UniversityMelbourne · Australia60 works · 54,866 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

Cite

Cite as Yuvan Ramesh and Tongtong Wu. 2026. Team Yuvan at SemEval-2026 Task 13: Task-Adaptive Ensemble Strategies for AI-Generated Code Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.263

@inproceedings{ramesh2026team,
  title = {{Team Yuvan at SemEval-2026 Task 13: Task-Adaptive Ensemble Strategies for AI-Generated Code Detection}},
  author = {Yuvan Ramesh and Tongtong Wu},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.263},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.263}
}

References

14 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aylin Caliskan-Islam, Richard Harang, Andrew Liu, Arvind Narayanan, Clare Voss, Fabian Yamaguchi, and Rachel Greenstadt. 2015. De-anonymizing programmers via code stylometry. In 24th USENIX Security Symposium (USENIX Security 15), pages 255–270. USENIX Association. find
  2. Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. 2019. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9268–9277. find
  3. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Code-BERT: A pre-trained model for programming and natural languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1536–1547,… find
  4. Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. GraphCodeBERT: Pre-training code representations with data…
  5. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR).
  6. Yuling Shi, Hongyu Zhang, Chengcheng Wan, and Xiaodong Gu. 2025. Between lines of code: Unraveling the distinct patterns of machine and human programmers. In Proceedings of the 47th International Conference on Software Engineering (ICSE). find
  7. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, and 5 others. 2024. Qwen2.5-Coder technical report. Preprint, arXiv:2409.12186.
  8. Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune BERT for text classification? In Proceedings of the 18th China National Conference on Computational Linguistics (CCL). Springer.
  9. Oseremen Joy Idialu, Noble Saji Mathews, Rungroj Maipradit, Joanne M. Atlee, and Mei Nagappan. 2024. Whodunit: Classifying code as human authored or GPT-4 generated – a case study on CodeChef problems. In Proceedings of the 21st International Conference on Mining Software Repositories (MSR), pages 3… find
  10. This appendix presents the full ablation studies for each task that were summarized in the main body. find
  11. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. DetectGPT: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning (ICML).
  12. Phuong T. Nguyen, Juri Di Rocco, Claudio Di Sipio, Riccardo Rubei, Davide Di Ruscio, and Massimiliano Di Penta. 2024. GPTSniffer: A CodeBERT-based classifier to detect source code written by ChatGPT. Journal of Systems and Software. find
  13. Daniil Orel, Dilshod Azizov, and Preslav Nakov. 2025a. CoDet-M4: Detecting machine-generated code in multi-lingual, multi-generator and multi-domain settings. In Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics. find
  14. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026a. AICD bench: A challenging benchmark for AI-generated code detection. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026). Associat…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ramesh-2026-team-yuvan-semeval-2026-task/