Elamur Research⌕
☰
home / papers / 2026 / DeSegMa-IT at EVALITA 2026: Overview of the “Detection and Segmentation of Machine Generated Texts in Italian”

DeSegMa-IT at EVALITA 2026: Overview of the “Detection and Segmentation of Machine Generated Texts in Italian”

Paper · 2026 · Giovanni Puccetti, Andrea Pedrotti and Andrea Esuli

Summary

DeSegMa-IT is an EVALITA 2026 shared task on detecting machine-generated text (MGT) in Italian, a language with few such tools compared with English. It tests whether detectors still work when the generators at test time differ from those seen in training. Sub-task A is binary classification of whole documents. Sub-task B asks systems to find the character where a human-written news passage turns into an LLM-written continuation.

The organisers built both datasets from Change-IT articles from La Repubblica and Il Giornale. For sub-task A, LLMs wrote synthetic articles from the original headlines, with prompts that mimicked each outlet's political leaning. Machine texts were then filtered by perplexity so they look closer to the human ones.

In sub-task A, detectors were accurate but not perfect, and they struggled most with LLMs pretrained from scratch on Italian. In sub-task B, the best systems fine-tuned Italian encoder models and framed the task as sequence labeling.

The benchmark is a reusable resource for evaluating Italian MGT detectors under realistic distribution shift.

Key points

  • Sub-task A data: 15,135 balanced training articles (selected from 33,138 collected) and 13,268 test articles. The training and test splits were generated by different pools of LLMs.
  • Perplexity filtering: machine texts whose perplexity strayed too far from the human mean were removed, using gemma-3-12b-it as the scoring model.
  • Recall by generator (sub-task A):
    • Above 90% for Llama-3.3-70B, Gemma-3-27B and ANITA-Next-24B.
    • Around 50% for models pretrained natively on Italian.
    • Llama-3.1-8B was an unexpected exception, evading some detectors despite being English-pretrained.
  • False positives: false positive rates ranged from 0.3% to 2%. The Pangram detector had 0 FPR but lower overall accuracy.
  • Sub-task B results:
    • Data: 19,945 training and 23,211 test samples from nine LLMs, with human prefixes of 64–512 characters.
    • The metric is character-level MAE. The best score was 52.54, from Stochastic Gradient Descenders using UmBERTo.
    • Only two of the five submitted systems beat the baseline.
    • SmolLM3-3B and Minerva 7B were hardest to segment, with average MAE above 70.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as Giovanni Puccetti, Andrea Pedrotti and Andrea Esuli. 2026. DeSegMa-IT at EVALITA 2026: Overview of the “Detection and Segmentation of Machine Generated Texts in Italian”. Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{puccetti2026desegma,
  title = {{DeSegMa-IT at EVALITA 2026: Overview of the “Detection and Segmentation of Machine Generated Texts in Italian”}},
  author = {Giovanni Puccetti, Andrea Pedrotti and Andrea Esuli},
  booktitle = {Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026)},
  year = {2026},
  url = {https://aclanthology.org/2026.evalita-1.5/}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. OpenAI, GPT-4 Technical Report, 2023. URL: https://arxiv.org/abs/2303.08774. ↗
  2. A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al., The llama 3 herd of models, arXiv preprint arXiv:2407.21783 (2024).
  3. DeepSeek-AI, Deepseek-v3 technical report, 2024. URL: https://arxiv.org/abs/2412.19437. arXiv:2412.19437 . ↗
  4. L. Dugan, D. Ippolito, A. Kirubarajan, S. Shi, C. Callison-Burch, Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text, Proceedings of the AAAI Conference on Artificial Intelligence 37 (2023) 12763–12771. URL: https://ojs.aaai.org/inde… — [doi:10.1609/aaai.v37i11.26501](https://doi.org/10.1609/aaai.v37i11.26501) ↗
  5. G. Puccetti, A. Rogers, C. Alzetta, F. Dell’Orletta, A. Esuli, AI ‘news’ content farms are easy to make and hard to detect: A case study in italian, in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Ling… find
  6. E. Kasneci, K. Sessler, S. Küchemann, M. Bannert, D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. Günnemann, E. Hüllermeier, S. Krusche, G. Kutyniok, T. Michaeli, C. Nerdel, J. Pfeffer, O. Poquet, M. Sailer, A. Schmidt, T. Seidel, M. Stadler, J. Weller, J. Kuhn, G. Kasneci, Chatgpt for good? on op… — ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
  7. M. Corazza, G. Longo, L. Zilli, E. Di Sante, S. Sapienza, M. Palmirani, Hybrid ai enhancing european drafting legislation for a better regulation, in: 2025 Eleventh International Conference on eDemocracy & eGovernment (ICEDEG), IEEE, 2025, pp. 106–113. find
  8. Y. Li, Q. Li, L. Cui, W. Bi, Z. Wang, L. Wang, L. Yang, S. Shi, Y. Zhang, MAGE: Machine-generated text detection in the wild, in: L.-W. Ku, A. Martins, V. Srikumar (Eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for…
  9. X. Hu, P.-Y. Chen, T.-Y. Ho, RADAR: robust ai-text detection via adversarial learning, in: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. find
  10. M. Abassy, K. Elozeiri, A. Aziz, M. N. Ta, R. V. Tomar, B. Adhikari, S. E. D. Ahmed, Y. Wang, O. Mohammed Afzal, Z. Xie, J. Mansurov, E. Artemova, V. Mikhailov, R. Xing, J. Geng, H. Iqbal, Z. M. Mujahid, T. Mahmoud, A. Tsvigun, A. F. Aji, A. Shelmanov, N. Habash, I. Gurevych, P. Nakov, LLM-DetectAIv… find
  11. A. Pedrotti, M. Papucci, C. Ciaccio, A. Miaschi, G. Puccetti, F. Dell’Orletta, A. Esuli, Stress-testing machine generated text detection: Shifting language models writing style to fool detectors, in: W. Che, J. Nabende, E. Shutova, M. T. Pilehvar (Eds.), Findings of the Association for Computational… find
  12. Y. Wang, J. Mansurov, P. Ivanov, J. Su, A. Shelmanov, A. Tsvigun, O. M. Afzal, T. Mahmoud, G. Puccetti, T. Arnold, Semeval-2024 task 8: Multidomain, multimodel and multilingual machine-generated text detection, in: A. K. Ojha, A. S. Dogruöz, H. T. Madabushi, G. D. S. Martino, S. Rosenthal, A. Rosá (… find
  13. Y. Wang, A. Shelmanov, J. Mansurov, A. Tsvigun, V. Mikhailov, R. Xing, Z. Xie, J. Geng, G. Puccetti, E. Artemova, J. Su, M. N. Ta, M. Abassy, K. A. Elozeiri, S. E. D. A. El Etter, M. Goloburda, T. Mahmoud, R. V. Tomar, N. Laiyk, O. Mohammed Afzal, R. Koike, M. Kaneko, A. F. Aji, N. Habash, I. Gurevy… find
  14. Y. Wang, J. Mansurov, P. Ivanov, J. Su, A. Shelmanov, A. Tsvigun, O. M. Afzal, T. Mahmoud, G. Puccetti, T. Arnold, A. F. Aji, N. Habash, I. Gurevych, P. Nakov, M4gt-bench: Evaluation benchmark for black-box machine-generated text detection, 2024. arXiv:2402.11175 . find
  15. F. Cutugno, A. Miaschi, A. P. Aprosio, G. Rambelli, L. Siciliani, M. A. Stranisci, Evalita 2026: Overview of the 9th evaluation campaign of natural language processing and speech tools for italian, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for I… find
  16. E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, C. Finn, Detectgpt: zero-shot machine-generated text detection using probability curvature, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023.
  17. L. D. Mattei, M. Cafagna, F. Dell’Orletta, M. Nissim, A. Gatt, CHANGE-IT @ EVALITA 2020: Change headlines, adapt news, generate, in: V. Basile, D. Croce, M. D. Maro, L. C. Passaro (Eds.), Proceedings of Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Wo… find
  18. Z. Xu, S. Escalera, A. Pavão, M. Richard, W.-W. Tu, Q. Yao, H. Zhao, I. Guyon, Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform, Patterns 3 (2022) 100543. URL: https://www.sciencedirect.com/science/article/pii/S2666389922001465. doi: https://doi.org/ 10.1016/j.patter.2022.1… — [doi:10.1016/j.patter.2022.100543](https://doi.org/10.1016/j.patter.2022.100543) ↗
  19. T. P. T. Nhan, B. H. Son1, D. V. Thin, Gradient descenders at DeSegMa-IT: Leveraging monolingual transformer for LLM-generated text detection and boundary identification, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVA…
  20. C. J. Scozzaro, M. Rinaldi, G. Mittone, M. A. Stranisci, Kenji-Endo: a BabyLM at EVALITA 2026, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026. find
  21. F. Borazio, G. D. Luca, D. Pasquini, D. Croce, R. Basili, UniTor at DeSegMa-IT Analyzing supervision and encoder representations for italian machine-generated text detection, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop… find
  22. L. D. Mattei, M. Cafagna, F. Dell’Orletta, M. Nissim, A. Gatt, CHANGE-IT @ EVALITA 2020: Change headlines, adapt news, generate (short paper), in: V. Basile, D. Croce, M. D. Maro, L. C. Passaro (Eds.), Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for… find
  23. K. Dubey, Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification, in: D. Hupkes, V. Dankers, K. Batsuren, A. Kazemnejad, C. Christodoulopoulos, M. Giulianelli, R. Cotterell (Eds.), Proceedings of the 2nd GenBench Workshop on Generalisation (Be… find
  24. N. Auletta, Nicla at DeSegMa-IT: DistilBERT for MGT detection and LightGBM regressor for HMT segmentation., in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026. find
  25. H. N. Phu, B. H. Son, D. V. Thin, Stochastic Gradient Descenders at DeSegMa-IT: Instruction-tuned LLM and token classification for MGT detection and boundary localization, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EV…
  26. L. Parisi, S. Francia, P. Magnani, UmBERTo: an italian language model trained with whole word masking, https://github.com/musixmatchresearch/umberto, 2020. ↗
  27. F. Giobergia, MINDS at DeSegMa-IT: Detecting human-LLM authorship switches via token-level classification, in: Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR.org, Bari, Italy, 2026. find
  28. P. He, J. Gao, W. Chen, DeBERTav3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing, in: The Eleventh International Conference on Learning Representations, 2023. URL: https://openreview.net/forum?id=sE7-XhLxHA. ↗
  29. B. Emi, M. Spero, Technical report on the pangram ai-generated text classifier, 2024. URL: https://arxiv.org/abs/2402.14873. arXiv:2402.14873 . — [arXiv:2402.14873](https://arxiv.org/abs/2402.14873) ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/puccetti-2026-desegma-it-evalita-2026-overview/