Elamur Research⌕
☰
home / papers / 2026 / UIT-AMMC at SemEval-2026 Task 13: Exploiting Structural Formatting Signatures for Robust AI-Generated Code Detection

UIT-AMMC at SemEval-2026 Task 13: Exploiting Structural Formatting Signatures for Robust AI-Generated Code Detection

Paper · 2026 · Cuong Pham, Minh Nguyen, Minh Le et al.

Summary

This system addresses SemEval-2026 Task 13 Subtask A: deciding whether a code snippet was written by a human or generated by AI. Training data cover only algorithmic code in a few languages. The test set, by contrast, introduces Research and Production domains along with languages never seen in training, namely Go, PHP, C#, C and JS.

The authors argue that semantic cues blur because humans and LLMs converge on similar solutions. Alignment tuning, however, leaves AI code with recognizable layout habits, especially a higher share of blank lines (R_void). Their Structure-Aware Contrastive Cascade works in four steps:

  • It computes layout, entropy and naming/line-length statistics.
  • It passes these as a short textual report, alongside the code, to a Qwen-2.5-Coder 14B model. The model was fine-tuned with QLoRA on a 270K subset of the official 500K corpus, with renaming, whitespace and comment-stripping augmentation.
  • It combines the model's contrastive likelihoods for both labels with a capped empty-line prior.
  • It sends low-confidence cases to an ACE-inspired three-agent debate.

The system scored Macro F1 0.802, placing 3rd of 82 teams. This suggests that cheap formatting cues can usefully support LLM-based code forensics under distribution shift.

Key points

  • Data: The official corpus has about 500K training samples (238K human, 262K AI). The team used 270K for training and 30K for validation, split by project to avoid stylistic leakage.
  • R_void as a signal:
    • On the training data the human/AI difference in R_void was significant (Welch t = 47.2, Cohen's d = 0.32).
    • On the test diagnostic subset the effect was larger (d = 0.97). Mean blank-line share was 7.25% for human code versus 16.50% for AI code.
  • Ablation (Macro F1):
    • R_void thresholding alone: 0.679
    • QLoRA SFT: 0.740
    • SFT with features in the prompt plus fusion (λ = 0.82): 0.785
    • Full cascade with agentic debate: 0.802 (bootstrap 95% CI 0.788–0.816)
  • Per language: C# scored highest (F1 0.842) and PHP lowest (0.452). In PHP, embedded HTML templates reverse the direction of the blank-line signal.
  • Limitations: The method may be vulnerable to adversarial reformatting. The error audit (100 samples, one annotator) found that false positives came mostly from educational code and boilerplate.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

Cite

Cite as Cuong Pham, Minh Nguyen, Minh Le, An Nguyen and Chinh Nguyen. 2026. UIT-AMMC at SemEval-2026 Task 13: Exploiting Structural Formatting Signatures for Robust AI-Generated Code Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.60

@inproceedings{pham2026uit,
  title = {{UIT-AMMC at SemEval-2026 Task 13: Exploiting Structural Formatting Signatures for Robust AI-Generated Code Detection}},
  author = {Cuong Pham, Minh Nguyen, Minh Le, An Nguyen and Chinh Nguyen},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.60},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.60}
}

References

29 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Jibi Ajiam, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Maor Ashkenazi, Shachar Langbeheim, Shai Shalev-Shwartz, and Amit Daniely. 2025. Zero-shot detection of llm-generated code via approximated task conditioning. arXiv preprint arXiv:2506.06069. Accepted at ECML-PKDD 2025. — [arXiv:2506.06069](https://arxiv.org/abs/2506.06069) find
  3. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2024. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In Proceedings of ICLR.
  4. Brett A. Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos. 2023. Programming is hard - or at least it used to be: Educational opportunities and challenges of ai code generation. In Proceedings of the 54th ACM Technical Symposium on Computer Scien… find
  5. Aylin Caliskan-Islam, Richard Harang, Andrew Liu, Arvind Narayanan, Clare Voss Joint, Fabian Yamaguchi, and Rachel Greenstadt. 2015. De-anonymizing programmers via code stylometry. In Proceedings of the 24th USENIX Security Symposium. find
  6. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. In Proceedings of NeurIPS.
  7. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Codebert: A pre-trained model for programming and natural languages. In Proceedings of EMNLP.
  8. Jiachen Fu, Siyuan Li, and 1 others. 2025. Detectanyllm: Towards generalizable and robust detection of machine-generated text. In Proceedings of ACM Multimedia (MM ’25). find
  9. Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, and 1 others. 2021. Graphcodebert: Pre-training code representations with data flow. In Proceedings of ICLR.
  10. Hanxi Guo, Siyuan Cheng, Kaiyuan Zhang, Bohong Wu, Siyuan Chen, and Zhidong Deng. 2025. Codemirage: A multi-lingual benchmark for detecting ai-generated and paraphrased source code. arXiv preprint arXiv:2506.11059. find
  11. Huyen Ha, Duc Tran, and Dukyun Kim. 2023. Black-box adversarial attacks against language model detector. In Proceedings of the 12th International Symposium on Information and Communication Technology (SoICT 2023). find
  12. Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamed Fendley, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: Zero-shot detection of machine-generated text. In Proceedings of ICML. find
  13. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, and 5 others. 2024. Qwen2.5-coder technical report. Preprint, arXiv:2409.12186.
  14. Siyuan Li, Jiachen Fu, Di Fu, Cheng-Chieh Huang, and 1 others. 2025. Styledecipher: Robust and explainable detection of llm-generated texts with stylistic analysis. arXiv preprint arXiv:2510.12608. — [arXiv:2510.12608](https://arxiv.org/abs/2510.12608) find
  15. Zeyu Ma, Hongzhan Chen, Hongbo Xu, Jianzheng Liu, Lingyong Yan, Wei He, and Xueqi Cheng. 2024. Conda: Contrastive domain adaptation for ai-generated text detection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6524–6538. find
  16. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning (ICML).
  17. Trieu Hai Nguyen and Sivaswamy Akilesh. 2025. Vietbinoculars: A zero-shot approach for detecting vietnamese llm-generated text. arXiv preprint arXiv:2509.26189. — [arXiv:2509.26189](https://arxiv.org/abs/2509.26189) find
  18. Daniil Orel, Dilshod Azizov, and Preslav Nakov. 2025a. Codet-m4: Detecting machine-generated code in multi-lingual settings. In Findings of the Association for Computational Linguistics: ACL 2025. find
  19. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026a. AICD bench: A challenging benchmark for ai-generated code detection. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Paper…
  20. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026b. SemEval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios. In Proceedings of the 20th International Workshop on Semantic Evaluati…
  21. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. 2025b. Droid: A resource suite for AI-generated code detection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31263–31289, Suzhou, China. Association for Computational Linguistics. find
  22. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, and 1 others. 2022. Training language models to follow instructions with human feedback. In Proceedings of NeurIPS.
  23. Wei Hung Pan, Cheng-Chieh Huang, Yan-Ling Chang, and Huan-Chao Keh. 2024. Assessing ai detectors in identifying ai-generated code: Implications for education. In Proceedings of the 2024 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE), pages 321–327. ACM. find
  24. Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. 2023. Do users write more insecure code with ai assistants? In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. find
  25. Hung Pham, Huyen Ha, Van Tong, Duc Tran, and Dukyun Kim. 2024. Magecode: Machine-generated code detection method using large language models. IEEE Access, 12:164230–164245. find
  26. Yuling Shi, Hongyu Zhang, Chengcheng Wan, and Xiaodong Gu. 2025. Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers, page 1628–1639. IEEE Press. find
  27. Xianjun Yang, Wei Cheng, Yue Wu, Linda Petzold, William Yang Wang, and Haifeng Chen. 2023. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. In Proceedings of EMNLP. find
  28. Yang Yang, Xinyu Li, Duyu Tang, and 1 others. 2025. Unicor: Modality collaboration for robust cross-language hybrid code retrieval. In Proceedings of ICSE 2026. find
  29. Qizheng Zhang, Changran Hu, Shubhangi Upasani, Shubham Toshniwal, Yashar Mehdad, Veselin Stoyanov, and Sewon Min. 2025. Agentic context engineering: Evolving contexts for self-improving language models. arXiv preprint arXiv:2510.04618. Accepted at ICLR 2026. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pham-2026-uit-ammc-semeval-2026-task/