Elamur Research⌕
☰
home / papers / 2026 / CEAID: Benchmark of Multilingual Machine-Generated Text Detection Methods for Central European Languages

CEAID: Benchmark of Multilingual Machine-Generated Text Detection Methods for Central European Languages

Paper · 2026 · Dominik Macko and Jakub Kopál

Summary

Research on detecting machine-generated text (MGT) has focused mostly on English. Detectors for Central European languages therefore depend on uncertain cross-lingual transfer. The authors build the first benchmark for this region, CEAID. It covers seven languages and two domains: news articles from MULTITuDE_v3 and social-media posts from MultiSocial. The machine texts come from eight different LLMs. The benchmark compares three families of detectors: zero-shot statistical, off-the-shelf pretrained, and models finetuned on the train split. It also tests which combinations of training languages generalise best to the rest of the region, and how well detectors hold up against paraphrasing and homoglyph obfuscation. Finetuned detectors clearly outperform the other two families in every tested language and are also the most resistant to obfuscation. The particular mix of training languages affects generalisation only slightly. The work matters because it gives under-served languages concrete, language-aware detectors and evidence about how to train them, instead of relying on English-centric tools.

Key points

  • Languages: The benchmark includes five Slavic languages (Czech, Polish, Croatian, Slovak, and Slovenian) alongside German and Hungarian. These span Germanic, Slavic, and Uralic branches, all in Latin script. Slovak and Slovenian are used only for testing because they have too little social-media data.
  • Dataset size: about 132k training and 56k test samples (Table 1).
  • Generators: six are shared across both domains: Aya-101, GPT-3.5-Turbo-0125, Mistral-7B-Instruct-v0.2, OPT-IML-Max-30B, v5-Eagle-7B-HF, and Vicuna-13B. Llama-2-70B-chat-hf appears only in news, and Gemini only in social media.
  • Detectors (10 in total):
    • Statistical: Binoculars, Fast-DetectGPT, and LLM-Deviation, built on mGPT.
    • Pretrained: ChatGPT-detector-RoBERTa-Chinese, Longformer Detector, and BLOOMZ-3B-mixed-detector.
    • Finetuned: e.g., Llama-3.2-3B, mDeBERTa-v3-base, Gemma-2-2B, and XLM-RoBERTa-base.
  • Robustness results: homoglyph attacks almost wipe out the statistical detectors (Fast-DetectGPT shows a reported change of about −90 in Table 7). Finetuned models such as Llama-3.2-3B change only marginally. Source code is available at github.com/kinit-sk/CEAID for non-commercial research.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Dominik Macko and Jakub Kopál. 2026. CEAID: Benchmark of Multilingual Machine-Generated Text Detection Methods for Central European Languages. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1458

@inproceedings{macko2026ceaid,
  title = {{CEAID: Benchmark of Multilingual Machine-Generated Text Detection Methods for Central European Languages}},
  author = {Dominik Macko and Jakub Kopál},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1458},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1458}
}

References

36 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dimosthenis Antypas, Asahi Ushio, Jose Camacho-Collados, Vitor Silva, Leonardo Neves, and Francesco Barbieri. 2022. Twitter topic classification. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3386– 3400, Gyeongju, Republic of Korea. International Committee o… find
  2. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. Fast-DetectGPT: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations.
  3. Robert Bideleux and Ian Jeffries. 2007. A history of Eastern Europe: Crisis and change. Routledge. find
  4. Matyas Bohacek. 2023. The unseen A+ student: Navigating the impact of large language models in the classroom. In Workshop on Challenges in Deployable Generative AI at International Conference on Machine Learning (ICML). find
  5. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… find
  6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. DeepSeek-R1: Incentivizing reasoning capability in LL…
  7. Liam Dugan, Alyssa Hwang, Filip Trhlík, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. RAID: A shared benchmark for robust evaluation of machine-generated text detectors. In Proceedings of the 62nd Annual Meeting of the Association for Computational Lingui…
  8. Pieter Fivez, Walter Daelemans, Tim Van de Cruys, Yury Kashnitsky, Savvas Chamezopoulos, Hadi Mohammadi, Anastasia Giachanou, Ayoub Bagheri, Wessel Poelman, Juraj Vladika, Esther Ploeger, Johannes Bjerva, Florian Matthes, and Hans van Halteren. 2024. The CLIN33 shared task on the detection of text g… find
  9. Paweł Gryka, Kacper Gradoń, Marek Kozłowski, Miłosz Kutyła, and Artur Janicki. 2024. Detection of ai-generated emails - a case study. In Proceedings of the 19th International Conference on Availability, Reliability and Security, ARES ’24, New York, NY, USA. Association for Computing Machinery. find
  10. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection. arXiv preprint arxiv:2301.07597.
  11. Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: zero-shot detection of machine-generated text. In Proceedings of the 41st International Conference on Machine Learning, ICML’2… find
  12. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2022. DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. In The Eleventh International Conference on Learning Representations.
  13. Verena Irrgang, Veronika Solopova, Steffen Zeiler, Robert M. Nickel, and Dorothea Kolossa. 2024. Features and detectability of German texts generated with large language models. In Proceedings of the 20th Conference on Natural Language Processing (KONVENS 2024), pages 264–280, Vienna, Austria. Assoc… find
  14. Taja Kuzman, Igor Mozetič, and Nikola Ljubešić. 2023. Automatic genre identification for robust enrichment of massive text collections: Investigation of classification methods in the era of large language models. Machine Learning and Knowledge Extraction, 5(3):1149–1175. find
  15. Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. MAGE: Machine-generated text detection in the wild. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 36–53, Bangk…
  16. Dominik Macko, Jakub Kopál, Robert Moro, and Ivan Srba. 2025a. MultiSocial: Multilingual benchmark of machine-generated text detection of social-media texts. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 727–752, Vienna, Aus…
  17. Dominik Macko, Jakub Kopal, Robert Moro, and Ivan Srba. 2025b. MULTITuDEv3. find
  18. Dominik Macko, Robert Moro, and Ivan Srba. 2025c. Increasing the robustness of the fine-tuned multilingual machine-generated text detectors. Preprint, arXiv:2503.15128. — [arXiv:2503.15128](https://arxiv.org/abs/2503.15128) find
  19. Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2023. MULTITuDE: Large-scale multilingual machine-generated text detection benchmark. In Proceedings of the 2023 Conference on Empirical Met… find
  20. Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason S Lucas, Michiharu Yamashita, Nafis Irtiza Tripto, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2024. Authorship obfuscation in multilingual machine-generated text detection. In Findings of the Association for Computational Linguistics: E… find
  21. AI Meta. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. Meta AI Blog. Retrieved December, 20:2024. find
  22. Areg Mikael Sarvazyan, José Ángel González, Marc Franco-Salvador, Francisco Rangel, Berta Chulvi, and Paolo Rosso. 2023. Overview of autextification at iberlef 2023: Detection and attribution of machine-generated text in multiple domains. Procesamiento del Lenguaje Natural, 71:275–288. find
  23. Areg Mikael Sarvazyan, José Ángel González, Francisco Rangel, Paolo Rosso, and Marc Franco-Salvador. 2024. Overview of iberautextification at IberLEF 2024: Detection and attribution of machine-generated text on languages of the Iberian peninsula. Procesamiento del Lenguaje Natural, (73):421–434. find
  24. Kristina Schaaff, Tim Schlippe, and Lorenz Mindner. 2023. Classification of human- and AI-generated texts for English, French, German, and Spanish. In Proceedings of the 6th International Conference on Natural Language and Speech Processing (ICNLSP 2023), pages 1–10, Online. Association for Computat… find
  25. Tatiana Shamardina, Vladislav Mikhailov, Daniil Cher-nianskii, Alena Fenogenova, Marat Saidov, Anastasiya Valeeva, Tatiana Shavrina, Ivan Smurov, Elena Tutubalina, and Ekaterina Artemova. 2022. Findings of the the RuATD shared task 2022 on artificial text detection in Russian. In Computational Lingu… find
  26. Oleh Shliazhko, Alena Fenogenova, Maria Tikhonova, Anastasia Kozlova, Vladislav Mikhailov, and Tatiana Shavrina. 2024. mGPT: Few-shot learners go multilingual. Transactions of the Association for Computational Linguistics, 12:58–79.
  27. Petr Šigut and Tomáš Foltỳnek. 2023. Can we detect ChatGPT-generated texts in Czech and Slovak languages? RASLAN 2023 Recent Advances in Slavonic Natural Language Processing, page 35. find
  28. Nicolai Thorer Sivesind and Andreas Bentzen Winje. 2023. Machine-generated text-detection by fine-tuning of language models. find
  29. Michal Spiegel and Dominik Macko. 2024. IMGTB: A framework for machine-generated text detection benchmarking. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 172–179, Bangkok, Thailand. Association for Computational… find
  30. Irina Temnikova, Iva Marinova, Silvia Gargova, Ruslana Margova, and Ivan Koychev. 2023. Looking for traces of textual deepfakes in Bulgarian on social media. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 1151–1161, Varna, Bulgaria. IN-CO… find
  31. Cem Üyük, Danica Rovó, Shaghayeghkolli Shaghayeghkolli, Rabia Varol, Georg Groh, and Daryna Dementieva. 2024. Crafting tomorrow’s headlines: Neural news generation and detection in English, Turkish, Hungarian, and Persian. In Proceedings of the Third Workshop on NLP for Positive Impact, pages 271–30… find
  32. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, and Thomas Arnold. 2024a. SemEval-2024 task 8: Multidomain, multimodel and multilingual machine-generated text detection. In Proceedings of the 18th Internati… find
  33. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024b. M4GTbench: Evaluation benchmark for black-box machine-generated text detecti… find
  34. Yuxia Wang, Artem Shelmanov, Jonibek Mansurov, Akim Tsvigun, Vladislav Mikhailov, Rui Xing, Zhuohan Xie, Jiahui Geng, Giovanni Puccetti, Ekaterina Artemova, Jinyan Su, Minh Ngoc Ta, Mervat Abassy, Kareem Ashraf Elozeiri, Saad El Dine Ahmed El Etter, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar,… find
  35. Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, Tomáš Foltỳnek, Jean Guerrero-Dib, Olumide Popoola, Petr Šigut, and Lorna Waddington. 2023. Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1):1–39. find
  36. Zhendong Wu and Hui Xiang. 2023. MFD: Multi-feature detection of LLM-generated text. PREPRINT (Version 1) available at Research Square. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/macko-2026-ceaid-benchmark-multilingual-machine-generated/