Elamur Research⌕
☰
home / papers / 2026 / Farhan Nafis Rayhan at SemEval-2026 Task 13: Supervised Contrastive Learning Approach with Gated Multiclass Decomposition Ensemble Architecture for Code Authorship Identification

Farhan Nafis Rayhan at SemEval-2026 Task 13: Supervised Contrastive Learning Approach with Gated Multiclass Decomposition Ensemble Architecture for Code Authorship Identification

Paper · 2026 · Farhan Rayhan and Fariska Ruskanda

Summary

The paper describes a system for SemEval-2026 Task 13 Subtask B. The task is to attribute a code snippet either to a human author or to one of 10 LLM families, such as DeepSeek-AI, Qwen, OpenAI and Gemma. The data covers 8 programming languages and is heavily imbalanced, so ranking uses Macro-F1.

The system has three stages:

  • Encoders. Each snippet is split into two inputs. Natural-language parts (comments, strings, identifiers) go to ModernBERT, and the stripped-down code structure goes to UniXcoder. Both encoders are fine-tuned with Supervised Contrastive Learning (SupCon).
  • Experts. The resulting embeddings feed shallow experts. Four of them each specialise in a group of generator families with similar style, a "One-vs-Some" decomposition. A fifth, an XGBoost "Human Shield", separates human from machine code.
  • Aggregation. A context-aware gated meta-learner combines the expert outputs.

Dropping ModernBERT and keeping only UniXcoder gave the strongest results. The paper reports a Macro-F1 of 0.31389 and 26th place, above the official CodeBERT baseline. The work matters because it shows that low-budget, mostly shallow ensembles built on contrastive embeddings can attribute code to model families.

Key points

  • Dataset: 500,000 training rows, 100,000 validation rows and 500,000 test rows (test without the language field). Snippets span Python, C, C++, C#, PHP, Go, Java and JavaScript across algorithmic, production and research domains. Human code makes up 88.42% of the training data, while Gemma accounts for only 0.39%.
  • Loss function: an early weighted cross-entropy setup with code encoders reached only about 0.08 Macro-F1 on the public test set. Switching to SupCon (768-D embeddings, 128-D projection head) brought large gains.
  • Shallow experts: Residual MLP, CatBoost, and Attentive Zero-Shot and Few-Shot Prototypical Heads, plus the XGBoost Human Shield.
  • Removing ModernBERT: this improved Macro-F1 by up to 0.013. ModernBERT's natural-language input acted mainly as noise.
  • Extra machine samples: after the competition, the authors added 25–150 samples per out-of-group machine class to each expert's data. In the refined system, correct OpenAI attributions rose from 1233 to 1789 and 01-ai detections from 80 to 291.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Bandung Institute of TechnologyBandung · Indonesia5 works · 324 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

Cite

Cite as Farhan Rayhan and Fariska Ruskanda. 2026. Farhan Nafis Rayhan at SemEval-2026 Task 13: Supervised Contrastive Learning Approach with Gated Multiclass Decomposition Ensemble Architecture for Code Authorship Identification. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.431

@inproceedings{rayhan2026farhan,
  title = {{Farhan Nafis Rayhan at SemEval-2026 Task 13: Supervised Contrastive Learning Approach with Gated Multiclass Decomposition Ensemble Architecture for Code Authorship Identification}},
  author = {Farhan Rayhan and Fariska Ruskanda},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.431},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.431}
}

References

23 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alon Arad and Saharon Rosset. 2025. Improving multi-class calibration through normalization-aware iso-Omer Aydin, Enis Karaarslan, Fatih Safa Erenay, and Nebojsa Bacanin. 2025. Generative ai in academic writing: A comparison of deepseek, qwen, chatgpt, gemini, llama, mistral, and gemma. Preprint, ar… — [arXiv:2503.04765](https://arxiv.org/abs/2503.04765) find
  2. Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, page 785–794. ACM. find
  3. Tianyu Gao, Xu Han, Zhiyuan Liu, and Maosong Sun. 2019. Hybrid attention-based prototypical networks for noisy few-shot relation classification. Proceedings of the AAAI Conference on Artificial Intelligence, 33:6407–6414. find
  4. Renhua Gu and Xiangfeng Meng. 2024. AISPACE at SemEval-2024 task 8: A class-balanced soft-voting system for detecting multi-generator machine-generated text. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 1476–1481, Mexico City, Mexico. Association for… find
  5. Daya Guo, Shuai Lu, Nan Duan, Yanlin Wang, Ming Zhou, and Jian Yin. 2022. Unixcoder: Unified cross-modal pre-training for code representation. Preprint, arXiv:2203.03850.
  6. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. Preprint, arXiv:1512.03385.
  7. Hasan M. S. Jaashan and Wagdi Rashad Ali Bin-Hady. 2025. Stylometric analysis of ai-generated texts: a comparative study of chatgpt and deepseek. Cogent Arts & Humanities, 12(1):2553162. find
  8. Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. 1991. Adaptive mixtures of local experts. Neural Computation, 3(1):79–87. find
  9. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2021. Supervised contrastive learning. Preprint, arXiv:2004.11362. — [arXiv:2004.11362](https://arxiv.org/abs/2004.11362) find
  10. Haoran Li, Yufan Guo, and Ruibo He. 2023. On the generalization of machine-generated text detectors: A survey and new perspectives. arXiv preprint arXiv:2312.03668. — [arXiv:2312.03668](https://arxiv.org/abs/2312.03668) find
  11. Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, Benjamin Bossan, and Marian Tietz. 2022. PEFT: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft. ↗
  12. Shakked Noy and Whitney Zhang. 2023. Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654):187–192. 3493 9 find
  13. Daniil Orel, Dilshod Azizov, and Preslav Nakov. 2025a. CoDet-m4: Detecting machine-generated code in multi-lingual, multi-generator and multi-domain settings. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10570–10593, Vienna, Austria. Association for Computational Lin… find
  14. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. SemEval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios. In Proceedings of the 20th International Workshop on Semantic Evaluatio…
  15. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. 2025b. Droid: A resource suite for ai-generated code detection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31251–31277. find
  16. Terence Parr. 2013. The Definitive ANTLR 4 Reference. Pragmatic Bookshelf. find
  17. Henry Pearce, Benjamin Ahmad, Benjamin McKenzie, and Dawn Song Be-Ac. 2022. Asleep at the keyboard? assessing the security risks of ai code generators. In Proceedings of the IEEE Symposium on Security and Privacy (SP). IEEE. find
  18. Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2019. Catboost: unbiased boosting with categorical features. Preprint, arXiv:1706.09516. — [arXiv:1706.09516](https://arxiv.org/abs/1706.09516) find
  19. Farhan Nafis Rayhan. 2026. Semeval-2026 task 13 feature dataset. Farhannr28/ NL-Separated-SemEval-2026-Task13. Hugging Face dataset. find
  20. Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical networks for few-shot learning. Preprint, arXiv:1703.05175. find
  21. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. Smarter, better, faster, longer: A modern bidirectional encoder for fast,…
  22. Xiaodan Xu, Chao Ni, Xinrong Guo, Shaoxuan Liu, Xiaoya Wang, Kui Liu, and Xiaohu Yang. 2025. Distinguishing LLM-generated from human-written code by contrastive learning. ACM Transactions on Software Engineering and Methodology, 34(4):1–31. find
  23. Danyang Zhang, Junhao Song, Ziqian Bi, Xinyuan Song, Yingfang Yuan, Tianyang Wang, Joe Yeong, and Junfeng Hao. 2025. Mixture of experts in large language models. Preprint, arXiv:2507.11181. 3494 10 — [arXiv:2507.11181](https://arxiv.org/abs/2507.11181) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rayhan-2026-farhan-nafis-rayhan-semeval-2026/