Elamur Research⌕
☰
home / papers / 2026 / LAFED at SemEval-2026 Task 13: Language-Agnostic Feature Engineering for Cross-Lingual AI-Generated Code Detection

LAFED at SemEval-2026 Task 13: Language-Agnostic Feature Engineering for Cross-Lingual AI-Generated Code Detection

Paper · 2026 · Juan David Villate

Summary

SemEval-2026 Task 13 (Subtask A) asks systems to decide whether a code snippet was written by a person or by an AI model. The test data also contains programming languages that never appear in training. Many surface cues reveal the language rather than the author, so detectors often fail when the language changes.

LAFED addresses this with hand-crafted features meant to be language-neutral:

  • structural "skeleton" measures, such as indentation depth and variability, control-flow density, and rough McCabe and Halstead complexity;
  • character and whitespace stylometry;
  • micro-style habits, such as operator spacing and blank-line usage.

A selection step keeps features that separate AI from human code within each language and penalizes features that predict the language itself. An XGBoost classifier, tuned together with the selection thresholds via Optuna, was trained on 500,000 Python, Java and C++ samples. It reached macro-F1 0.75209 on the hidden evaluation and placed 5th. The work shows that cheap, interpretable features transfer reasonably across languages, although Go and PHP remain hard.

Key points

  • Data. The training set has 500,000 samples (52.3% AI). The 1,000-sample test set covers 8 languages (22.3% AI), and C, C#, Go, JavaScript and PHP are unseen in training. The selection step kept 214 of 286 candidate features. Optuna ran 700 trials with 5-fold stratified CV.
  • Overfitting gap. The best trial (243) scored OOF macro-F1 0.9740 but 0.7570 on the test set. The authors attribute this 0.2170 gap to combined shifts in language, domain and generator.
  • Per-language F1. C# scored 0.7753 and JavaScript 0.7683. Go scored 0.6400, with high precision (0.8929) but low recall. PHP scored 0.5238. Seen languages averaged 0.7458 and unseen ones 0.6797. Errors were linked to gofmt formatting in Go and embedded HTML in PHP.
  • Most useful features. Whitespace cues dominate: 8 of the top-10 features by XGBoost gain, led by interactions of the tab/space mixing ratio. Blank-line micro-features transfer best, while many character n-grams leak language identity.
  • Negative results. Isotonic calibration lowered the score to 0.7468, and label-shift correction gave no gain. Mixup-style augmentation, linear language-signal projections and group DRO all hurt performance (ΔF1 from −0.007 to −0.026). The 17-feature baseline scored 0.7180. Code is available at github.com/DavidVilem/LAFED-SemEval2026.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Juan David Villate. 2026. LAFED at SemEval-2026 Task 13: Language-Agnostic Feature Engineering for Cross-Lingual AI-Generated Code Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.9

@inproceedings{villate2026lafed,
  title = {{LAFED at SemEval-2026 Task 13: Language-Agnostic Feature Engineering for Cross-Lingual AI-Generated Code Detection}},
  author = {Juan David Villate},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.9},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.9}
}

References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. H1: Whitespace provides a universal discriminative signal. Supported. Top-ranked features are dominated by whitespace cues, especially tab_space_mix_ratio and its interactions. find
  2. Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794. find
  3. H2: Structural metrics (McCabe/Halstead) help cross-language detection. Partially supported. Skeletal features carry signal but are also more predictive of language, requiring leakage-aware selection. find
  4. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Tetsuya Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2623–2631.
  5. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios. In Proceedings of the 20th International Workshop on Semantic Evaluatio…
  6. Zhenyu Xu and Victor S. Sheng. 2024. Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38. find
  7. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. AICD Bench: A Challenging Benchmark for AI-Generated Code Detection. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers…
  8. M. Stone. 1974. Cross-Validatory Choice and Assessment of Statistical Predictions. Journal of the Royal Statistical Society. Series B (Methodological), 36(2):111–147. find
  9. Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018. mixup: Beyond Empirical Risk Minimization. In Proceedings of the 6th International Conference on Learning Representations (ICLR).
  10. Olivier Bousquet and André Elisseeff. 2002. Stability and Generalization. Journal of Machine Learning Research, 2:499–526. find
  11. Thomas J. McCabe. 1976. A Complexity Measure. IEEE Transactions on Software Engineering, SE-2(4):308–320. find
  12. Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. 2020. Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization. In Proceedings of the 8th International Conference on Learning Representations (ICLR). find
  13. Maurice H. Halstead. 1977. Elements of Software Science. Elsevier North-Holland. find
  14. G. Ann Campbell. 2017. Cognitive Complexity: A new way of measuring understandability. SonarSource Technical Report. find
  15. Yanai Elazar and Yoav Goldberg. 2018. Adversarial Removal of Demographic Attributes from Text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 11– 21. find
  16. Efstathios Stamatatos. 2013. On the Robustness of Authorship Attribution Based on Character N-gram Features. Journal of Law and Policy, 21. find
  17. Yunita Sari, Mark Stevenson, and Andreas Vlachos. 2017. Continuous N-gram Representations for Authorship Attribution. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 267–273. find
  18. Bianca Zadrozny and Charles Elkan. 2002. Transforming Classifier Scores into Accurate Multiclass Probability Estimates. In Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 694–699. find
  19. Alexandru Niculescu-Mizil and Rich Caruana. 2005. Predicting Good Probabilities with Supervised Learning. In Proceedings of the 22nd International Conference on Machine Learning. find
  20. Marco Saerens, Patrice Latinne, and Christine Decaestecker. 2002. Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure. Neural Computation, 14(1):21–41. find
  21. Isabelle Guyon and André Elisseeff. 2003. An Introduction to Variable and Feature Selection. Journal of Machine Learning Research, 3:1157–1182. find
  22. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. In Proceedings of the 40th International Conference on Machine Learning (ICML).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/villate-2026-lafed-semeval-2026-task-13/