Elamur Research⌕
☰
home / papers / 2026 / SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection

SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection

Paper · 2026 · Udaythalavesh S, Rajalakshmi Sivanaiah and Angel Deborah S

Summary

This paper describes team UdayThalavesh's system for SemEval-2026 Task 13 Subtask A, a binary task that separates human-written code from AI-generated code. The data come from AICD-Bench, which spans six programming languages and several generators. The authors fine-tuned microsoft/codebert-base on a class-balanced 200K subset drawn from about 500K training examples, which are roughly 65% human-written. Training ran on TPUs. After each epoch they searched for the decision threshold that maximised validation F1-macro. They also added a deterministic label flip step that inverts predictions after thresholding, meant to offset the model's skewed, overconfident probabilities. On validation the system reached F1-macro 0.874. On the hidden test set it scored only 0.53, ranking around 40th. The authors' error analysis attributes this drop to distribution shift across languages and generators, Python over-representation, and thresholds overfitted to validation. The paper's main value is as a cautionary case: decision-level tuning that helps in-distribution may not transfer to unseen code sources.

Key points

  • The model is microsoft/codebert-base (125M parameters), pretrained on CodeSearchNet, with a linear 768→2 head and 384-token inputs. Training used AdamW (lr 2e-5, weight decay 0.01) on TPUv3-8 with a global batch of 256 for 3 epochs.
  • Balanced sampling took 100K human and 100K AI examples but did not control for programming language.
  • The best validation threshold fell from 0.61 to 0.52 over the three epochs, with the threshold grid spanning 0.25–0.73.
  • Ablation results on validation F1-macro:
    • removing the label flip: 0.765
    • removing balancing: 0.812
    • using a fixed 0.5 threshold: 0.847
    • training on the full 500K set: 0.836
    • replacing CodeBERT with BERT-base: 0.791
    • replacing CodeBERT with DeBERTa-v3: 0.862
  • Validation reached 0.874, but the private leaderboard score was 0.53, ranking around 40th of more than 80 teams. Proposed future work includes CodeT5+, StarCoder, PEFT, and ensembles.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Sri Sivasubramaniya Nadar College of EngineeringChennai · India10 works · 9 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

Cite

Cite as Udaythalavesh S, Rajalakshmi Sivanaiah and Angel Deborah S. 2026. SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.132

@inproceedings{s2026semeval,
  title = {{SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection}},
  author = {Udaythalavesh S, Rajalakshmi Sivanaiah and Angel Deborah S},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.132},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.132}
}

References

17 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios. In Proceedings of the 20th International Workshop on Semantic Evaluatio…
  2. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. AICDBench: A Challenging Benchmark for AI-Generated Code Detection. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics. find
  3. Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Code-BERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of EMNLP 2020. find
  4. Hamel Husain et al. 2019. CodeSearchNet challenge: Evaluating the state of semantic code search. arXiv:1909.09436.
  5. Yue Wang, Weishi Wang, Shafiq Joty, and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. In Proceedings of ACL 2021. find
  6. Daya Guo, Shuai Lu, Nanyang Liu, Shengyu Fu, Shuo Ren, Alexey Svyatkovskiy, and Neel Sundaresan. 2022. find
  7. UniXCoder: Unified Multilingual Code Pre-training Transformer. In EMNLP 2022. find
  8. Yue Wang, Hung Le, Sonse Shakhawat, Zhe Gan, Ahmed H. Awadallah, and Hao Peng. 2023. CodeT5+: Open Code Large Language Models for Code Understanding and Generation. arXiv:2305.07922. find
  9. Raymond Li et al. 2023. StarCoder: may the source be with you! arXiv:2305.06161. find
  10. Loubna Ben Allal et al. 2023. SantaCoder: don’t reach for the stars! arXiv:2303.13424.
  11. Eric Mitchell, Yoon Kim, Alexander W. Berglund, and Chelsea Finn. 2023. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. In ICLR 2023.
  12. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A Watermark for Large Language Models. In ICML 2023.
  13. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. 2025. Droid: A Resource Suite for AI-Generated Code Detection. In Proceedings of EMNLP 2025. find
  14. Marcin Buda, Atsuto Maki, and Maciej A. Zwolinski. 2018. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106:249–259. find
  15. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In ICML. find
  16. Zachary C. Lipton, Charles Elkan, and Balakrishnan Narayanaswamy. 2014. Optimal Thresholding of Classifiers to Maximize F1 Score. In Proceedings of ECML PKDD. find
  17. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/s-2026-semeval-2026-task-13-fine/