Elamur Research⌕
☰
home / papers / 2025 / Exploring the Limitations of Detecting Machine-Generated Text

Exploring the Limitations of Detecting Machine-Generated Text

Paper · 2025 · Jad Doughman, Osama Mohammed Afzal, Hawau Olamide Toyin et al.

Summary

Detectors of machine-generated text (MGT) often report strong accuracy, but they are usually tested on data resembling their training data. This paper asks how well such detectors hold up when the writing style, domain, or complexity of the input changes. The authors test four detectors: a feature-based LR-GLTR model, two RoBERTa variants (RoBERTa-M4 and the OpenAI Detector), and zero-shot Llama-3.1-8B. They use three datasets: M4 (English portion), OUTFOX student essays, and IDMGSP scientific abstracts. They split each test set into subsets using measures computed with spaCy, including part-of-speech ratios, named-entity and object ratios, sentence length, Flesch Reading Ease, and Hapax/Dihapax lexical diversity. They then compare performance across these subsets and use SHAP to explain the RoBERTa results. Performance changes sharply with style. Some subsets push detectors to chance level or worse. Simple, easy-to-read texts are misclassified most often, while dense, difficult texts are handled well. This suggests that published accuracy figures overstate real-world reliability, which matters for uses such as checking student work for plagiarism.

Key points

  • Out-of-domain drops (macro F1):
    • RoBERTa trained on ArXiv falls from 0.99 to 0.33 on IDMGSP, a 66% drop.
    • LR-GLTR (ArXiv) falls from 0.95 to 0.60 on OUTFOX and to 0.53 on IDMGSP.
    • The OpenAI Detector falls from 0.95 to 0.80.
    • Zero-shot Llama-3.1-8B scores 0.50 on IDMGSP.
  • Sensitivity to surface features: As the share of adverbs grows, LR-GLTR's F1 collapses from 0.4 to 0.0 on one text class and from 0.6 to 0.3 on the other. When named entities, objects, or sentence length increase, the model stops recognizing human-written text.
  • Punctuation reliance: SHAP analysis shows that RoBERTa leans heavily on punctuation and whitespace. This points to overfitting and makes the model vulnerable to simple adversarial edits.
  • Readability and named entities:
    • Detection of MGT gets worse as Flesch-measured difficulty decreases.
    • Llama-3.1-8B more often misclassifies abstracts with few named entities (Welch's t = −2.289, df = 1680, p = 0.022).
  • Deployment risks: The authors flag adversarial manipulation, unfair treatment of particular writing styles, data drift as new LLMs appear, and domain shift. They call for evaluation on data from the specific domain where a detector will be used.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom359 works · 77,352 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Jad Doughman, Osama Mohammed Afzal, Hawau Olamide Toyin, Shady Shehata, Preslav Nakov and Zeerak Talat. 2025. Exploring the Limitations of Detecting Machine-Generated Text. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{doughman2025exploring,
  title = {{Exploring the Limitations of Detecting Machine-Generated Text}},
  author = {Jad Doughman, Osama Mohammed Afzal, Hawau Olamide Toyin, Shady Shehata, Preslav Nakov and Zeerak Talat},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.288/}
}

References

21 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohamed Hesham Ibrahim Abdalla, Simon Malberg, Daryna Dementieva, Edoardo Mosca, and Georg Groh. 2023. A benchmark dataset to distinguish human-written and machine-generated scientific papers. Information, 14(10). find
  2. Rudolph Flesch. 1948. A new readability yardstick. Journal of Applied Psychology, 32(3):221–233. find
  3. Leon Fröhling and Arkaitz Zubiaga. 2021. Feature-based detection of automated language models: tackling gpt-2, gpt-3 and grover. PeerJ Computer Science, 7:e443. find
  4. Raghav Gaggar, Ashish Bhagchandani, and Harsh Oza. 2023. Machine-generated text detection using deep learning. find
  5. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116, Florence, Italy. Association for Computat…
  6. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597.
  7. Zhen Guo and Shangdi Yu. 2023. Authentigpt: Detecting machine-generated text via black-box language models denoising. find
  8. Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: Zero-shot detection of machine-generated text. arXiv preprint arXiv:2401.12070. find
  9. Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023. Mgtbench: Benchmarking machine-generated text detection. arXiv preprint arXiv:2303.14822. — [arXiv:2303.14822](https://arxiv.org/abs/2303.14822) find
  10. Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. find
  11. Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks Lakshmanan. 2022. Automatic detection of entity-manipulated text using factual knowledge. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 86–93, Dublin, Ireland. Association for… find
  12. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. A watermark for large language models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 17061–17084. PMLR.
  13. Ryuto Koike, Masahiro Kaneko, and Naoaki Okazaki. 2024. Outfox: Llm-generated essay detection through in-context learning with adversarially generated examples. Proceedings of the AAAI Conference on Artificial Intelligence, 38(19):21258–21266. find
  14. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692.
  15. Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st international conference on neural information processing systems, NIPS’17, pages 4768–4777, Red Hook, NY, USA. Curran Associates Inc. Number of pages: 10 Place: Long Beach, Califo…
  16. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature.
  17. Nuzhat Prova. 2024. Detecting ai generated text based on nlp and machine learning approaches. find
  18. Alec Radford and Jeff Wu. 2019. [link]. find
  19. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. 2019. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203.
  20. Ruixiang Tang, Yu-Neng Chuang, and Xia Hu. 2023. The science of detecting llm-generated texts. find
  21. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024. M4: Multi-generator, multi-domain, and multilingual black-box ma… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/doughman-2025-exploring-limitations-detecting-machine-generated/