Elamur Research⌕
☰
home / papers / 2026 / TeamOmega at SemEval-2026 Task 13: Frozen vs. Trainable Representations for Out-of-Distribution AI-Generated Code Detection: A CodeBERT Fine-Tuning Study

TeamOmega at SemEval-2026 Task 13: Frozen vs. Trainable Representations for Out-of-Distribution AI-Generated Code Detection: A CodeBERT Fine-Tuning Study

Paper · 2026 · Nahid Niyaz Shovon and Md. Naim Parvez

Summary

The paper reports on Team_Omega's entry for Subtask A of SemEval-2026 Task 13. The subtask is a binary decision: was a code snippet written by a human or produced by a machine? The difficulty comes from the test set. Training data covers only algorithmic problems in C++, Python and Java. The hidden test set goes further, bringing in programming languages absent from training (Go, PHP, C#, C, JavaScript) along with code from domains not seen in training, namely research and production code.

The authors compare two CodeBERT setups:

  • Frozen encoder: the whole encoder stays fixed, and only a small MLP head on the [CLS] vector is trained.
  • Partial unfreezing: the final encoder layer is trained with a lower learning rate, under a deeper residual head that uses GELU, LayerNorm and class-weighted loss.

Each model saw the 500K training samples once. Adapting the encoder raised in-domain validation macro F1 considerably, but on the out-of-distribution leaderboard it did much worse. There it collapsed toward labelling nearly everything as AI-generated. The frozen model scored lower in-domain yet held up better under the shift.

The takeaway is that strong in-domain scores can be misleading for code-origin detection. Keeping pretrained representations intact may transfer better across languages than aggressive task adaptation.

Key points

  • Data: 500,000 training samples (238,475 human, 261,525 AI), about 100K for validation, and a 500K hidden test set. A 1,000-example labeled sample was used only for sanity checks.
  • Configuration A: frozen microsoft/codebert-base with a head of about 214K parameters (768→256→64→2, ReLU).
  • Configuration B: layer 11 unfrozen, with about 8.2M trainable parameters. The encoder learning rate was 2×10⁻⁵ and the head's was 2×10⁻⁴.
  • Results (validation → test macro F1): A went from 0.9216 to 0.5132, while B went from 0.9841 to 0.3026. The team placed 44th of 81 in Subtask A.
  • Error analysis on the 1,000-sample check: B reached 0.9821 recall on AI code but only 0.1287 on human code.
  • Caveats noted by the authors: B may have been undertrained after a single epoch, a limit set by Kaggle's 12-hour GPU budget on a P100. Class weighting is a confounding difference between the setups. They suggest contrastive learning or adversarial domain adaptation as next steps.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Rajshahi University of Engineering and TechnologyRajshahi · Bangladesh3 works

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

Cite

Cite as Nahid Niyaz Shovon and Md. Naim Parvez. 2026. TeamOmega at SemEval-2026 Task 13: Frozen vs. Trainable Representations for Out-of-Distribution AI-Generated Code Detection: A CodeBERT Fine-Tuning Study. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.255

@inproceedings{shovon2026teamomega,
  title = {{TeamOmega at SemEval-2026 Task 13: Frozen vs. Trainable Representations for Out-of-Distribution AI-Generated Code Detection: A CodeBERT Fine-Tuning Study}},
  author = {Nahid Niyaz Shovon and Md. Naim Parvez},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.255},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.255}
}

References

21 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Wasi U Ahmad, Snigdha Chakraborty, and 1 others. 2024. Out-of-distribution generalization for code authorship attribution. In Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics. find
  2. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
  3. Mark Chen, Jerry Tworek, Heewoon Jun, Qiming Yuan, Henrique Peneder Pinto, Jared Kaplan, Harri Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  4. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Code-BERT: A pre-trained model for programming and natural languages. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1536–1547. find
  5. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. find
  6. Daya Guo, Shuo Ren, Shuai Lu, Shengdong Li, Duyu Feng, Dazhao Liu, Ming Li, Ming Zhou, Nan Duan, and Alex Zhou. 2022. UniXcoder: Unified cross-modal pre-training for code representation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers…
  7. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  8. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. find
  9. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, and 1 others. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences. find
  10. Clara Kübler, Pasquale Minervini, and Sebastian Riedel. 2024. Semeval-2024 task 1: Track A (human vs. AI text detection). In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). Association for Computational Linguistics. find
  11. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  12. Eric Mitchell, Jingfeng Peng, Timo Dettmers, Daniel Krasnopolsky, Elaine A. Chi, Christopher Potts, and Eric Horvitz. 2023. DetectGPT: Zero-shot machine-generated text detection using probability curvature. Transactions on Machine Learning Research.
  13. Erik Nijkamp, Bo Li, Mark Rabe, and 1 others. 2023. Codegen: Open large language models for code. In International Conference on Learning Representations. find
  14. OpenAI. 2023. OpenAI classifier. https://openai.com/blog/new-ai-classifier-for-indicatin g-ai-written-text. Accessed: 2026-03-01. ↗
  15. Daniil Orel, Dilshod Azizov, and Preslav Nakov. 2025a. CoDet-M4: Detecting machine-generated code in multi-lingual, multi-generator and multi-domain settings. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10570–10593. find
  16. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. SemEval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios. In Proceedings of the 20th International Workshop on Semantic Evaluatio…
  17. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. 2025b. Droid: A resource suite for AI-generated code detection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31263–31289. find
  18. Shuo Ren, Daya Guo, Shuai Lu, Shengdong Li, Duyu Feng, Dazhao Liu, Ming Li, Ming Zhou, Nan Duan, and Alex Zhou. 2023. Cross-lingual code representation learning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Comput… find
  19. Leslie N. Smith. 2018. A disciplined approach to neural network hyper-parameters: Part 1 – learning rate, batch size, momentum, and weight decay. arXiv preprint arXiv:1803.09820. — [arXiv:1803.09820](https://arxiv.org/abs/1803.09820) find
  20. Yue Wang, Weishi Li, Marek Mishra, Shomir Goldwasser, Ding Wang, and Fei Xia. 2021. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Inter… find
  21. Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. 2023. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. arXiv preprint arXiv:2303.17568.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shovon-2026-teamomega-semeval-2026-task-13/