Elamur Research⌕
☰
home / papers / 2026 / Gradient Descenders at DeSegMa-IT: Leveraging Monolingual Transformer for LLM-Generated Text Detection and Boundary Identification

Gradient Descenders at DeSegMa-IT: Leveraging Monolingual Transformer for LLM-Generated Text Detection and Boundary Identification

Paper · 2026 · Tran Phuoc Thanh Nhan, Bui Hong Son and Dang Van Thin

Summary

The authors (team Gradient Descenders) took part in the DeSegMa-IT shared task at EVALITA 2026, which covers machine-generated text (MGT) in Italian. The task has two parts. Subtask A asks whether a text was written by a human or by an LLM. Subtask B uses texts that begin with human writing and continue with LLM output, and the goal is to find the token where authorship changes.

The team argues that monolingual Italian encoders capture the language's morphology better than multilingual models such as XLM-RoBERTa or mDeBERTa. For Subtask A, they fine-tuned UmBERTo as a sequence classifier. For Subtask B, they cast boundary finding as binary token labelling, using DeBERTa-base-Italian with a linear head. This gives a finer resolution than earlier sentence-level methods.

On the official test sets, the system ranked first in Subtask A and third in Subtask B. The results suggest that compact, language-specific encoders remain strong tools for detecting LLM text in languages other than English.

Key points

  • Subtask A: UmBERTo, a RoBERTa-style model pre-trained on the Italian portion of OSCAR, ranked 1st with 94.58% accuracy. Its classifier uses the [CLS] vector and a two-layer MLP (Tanh activation, dropout), trained with cross-entropy loss.
  • Subtask B backbone: DeBERTa-base-Italian is derived from mDeBERTa-v3-base. It was adapted to Italian by swapping the embedding layer and continuing pre-training on Italian data.
  • Subtask B results: the system ranked 3rd with an MAE of 62.66, ahead of Nicla by 39.38 and UniTor by 18.94. In the backbone ablation, mDeBERTa-V3 scored 63.34, slightly worse.
  • Limitations: the system is sensitive to domain shift, predicts each token's label independently, and truncates inputs at 512 tokens.
  • Planned work: Domain-Adversarial Neural Networks (DANN) for domain robustness, Bi-LSTM layers to model label dependencies, and Longformer or sliding-window inference for longer documents.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as Tran Phuoc Thanh Nhan, Bui Hong Son and Dang Van Thin. 2026. Gradient Descenders at DeSegMa-IT: Leveraging Monolingual Transformer for LLM-Generated Text Detection and Boundary Identification. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{nhan2026gradient,
  title = {{Gradient Descenders at DeSegMa-IT: Leveraging Monolingual Transformer for LLM-Generated Text Detection and Boundary Identification}},
  author = {Tran Phuoc Thanh Nhan, Bui Hong Son and Dang {Van Thin}},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.6/}
}

References

26 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, G. Lample, Llama: Open and efficient foundation language models, 2023. URL: https://arxiv.org/abs/2302.13971. arXiv:2302.13971 . ↗
  2. G. Puccetti, A. Pedrotti, A. Esuli, Desegma-it at evalita 2026: Overview of the detection and segmentation of machine generated text in italian task, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org,… find
  3. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… find
  4. A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, V. Stoyanov, Unsupervised cross-lingual representation learning at scale, 2020. URL: https://arxiv.org/abs/1911.02116. arXiv:1911.02116 . ↗
  5. P. He, J. Gao, W. Chen, Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing, 2021. arXiv:2111.09543 .
  6. L. Parisi, S. Francia, P. Magnani, Umberto: an italian language model trained with whole word masking, https://github.com/musixmatchresearch/umberto, 2020. ↗
  7. Osiria, Deberta-base-italian: Italian deberta model, https://huggingface.co/osiria/ deberta-base-italian, 2023. Pre-trained Italian language model based on DeBERTa architecture. ↗
  8. A. Abdaoui, C. Pradel, G. Sigel, Load what you need: Smaller versions of multilingual bert, 2020. URL: https://arxiv.org/abs/2010.05609. arXiv:2010.05609 . — [arXiv:2010.05609](https://arxiv.org/abs/2010.05609) ↗
  9. H.-Q. Nguyen-Son, N.-D. T. Tieu, H. H. Nguyen, J. Yamagishi, I. E. Zen, Identifying computer-generated text using statistical analysis, in: 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2017, pp. 1504–1511. doi: 10.1109/ APSIPA.2017.828227… find
  10. S. Gehrmann, H. Strobelt, A. M. Rush, Gltr: Statistical detection and visualization of generated text, 2019. URL: https://arxiv.org/abs/1906.04043. arXiv:1906.04043 . ↗
  11. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, Roberta: A robustly optimized bert pretraining approach, 2019. URL: https://arxiv.org/abs/1907. 11692. arXiv:1907.11692 . ↗
  12. I. Solaiman, M. Brundage, J. Clark, A. Askell, A. Herbert-Voss, J. Wu, A. Radford, G. Krueger, J. W. Kim, S. Kreps, M. McCain, A. Newhouse, J. Blazakis, K. McGuffie, J. Wang, Release strategies and the social impacts of language models, 2019. URL: https://arxiv.org/abs/1908.09203. arXiv:1908.09203 . ↗
  13. Y. Chen, H. Kang, V. Zhai, L. Li, R. Singh, B. Raj, Token prediction as implicit classification to identify LLM-generated text, in: H. Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics,… find
  14. R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, Y. Choi, Defending against neural fake news, 2020. URL: https://arxiv.org/abs/1905.12616. arXiv:1905.12616 . ↗
  15. P. Wang, L. Li, K. Ren, B. Jiang, D. Zhang, X. Qiu, Seqxgpt: Sentence-level ai-generated text detection, 2023. URL: https://arxiv.org/abs/2310.08903. arXiv:2310.08903 . ↗
  16. J. Devlin, M. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding, CoRR abs/1810.04805 (2018). URL: http://arxiv.org/abs/1810.04805. arXiv:1810.04805 . ↗
  17. L. Martin, B. Muller, P. J. Ortiz Suarez, Y. Dupont, L. Romary, E. de la Clergerie, D. Seddah, B. Sagot, Camembert: a tasty french language model, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, 2020, pp. 7203–72…
  18. L. Dugan, D. Ippolito, A. Kirubarajan, S. Shi, C. Callison-Burch, Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text, 2022. URL: https://arxiv.org/abs/2212.12672. arXiv:2212.12672 . — [arXiv:2212.12672](https://arxiv.org/abs/2212.12672) ↗
  19. Z. Zeng, L. Sha, Y. Li, K. Yang, D. Gašević, G. Chen, Towards automatic boundary detection for human-ai collaborative hybrid essay in education, 2023. URL: https://arxiv.org/abs/2307.12267. arXiv:2307.12267 . — [arXiv:2307.12267](https://arxiv.org/abs/2307.12267) ↗
  20. I. Loshchilov, F. Hutter, Decoupled weight decay regularization, 2019. URL: https://arxiv.org/abs/ 1711.05101. arXiv:1711.05101 . ↗
  21. A. Yang, B. Yang, B. Hui, B. Zheng, B. Yu, C. Zhou, C. Li, C. Li, D. Liu, F. Huang, G. Dong, H. Wei, H. Lin, J. Tang, J. Wang, J. Yang, J. Tu, J. Zhang, J. Ma, J. Yang, J. Xu, J. Zhou, J. Bai, J. He, J. Lin, K. Dang, K. Lu, K. Chen, K. Yang, M. Li, M. Xue, N. Ni, P. Zhang, P. Wang, R. Peng, R. Men,… find
  22. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, Lora: Low-rank adaptation of large language models, 2021. URL: https://arxiv.org/abs/2106.09685. arXiv:2106.09685 . ↗
  23. C. Sutton, A. McCallum, An introduction to conditional random fields, 2010. URL: https://arxiv.org/abs/1011.4088. arXiv:1011.4088 . — [arXiv:1011.4088](https://arxiv.org/abs/1011.4088) ↗
  24. P. Zhou, W. Shi, J. Tian, Z. Qi, B. Li, H. Hao, B. Xu, Attention-based bidirectional long short-term memory networks for relation classification, in: K. Erk, N. A. Smith (Eds.), Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Associat…
  25. Z. Guo, K. Jiao, X. Yao, Y. Wan, H. Li, B. Xu, L. Zhang, Q. Wang, Y. Zhang, Z. Mao, USTC-BUPT at SemEval-2024 task 8: Enhancing machine-generated text detection via domain adversarial neural networks and LLM embeddings, in: A. K. Ojha, A. S. Doğruöz, H. Tayyar Madabushi, G. Da San Martino, S. Rosent… find
  26. Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, V. Lempitsky, Domain-adversarial training of neural networks, 2016. URL: https://arxiv.org/abs/1505.07818. arXiv:1505.07818 . — [arXiv:1505.07818](https://arxiv.org/abs/1505.07818) ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nhan-2026-gradient-descenders-desegma-it-leveraging/