Elamur Research⌕
☰
home / papers / 2026 / Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI

Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI

Paper · 2026 · Yuxia Wang, Rui Xing, Jonibek Mansurov et al.

Summary

Earlier work suggested that people can barely tell LLM output from human writing. Most of that evidence came from English text, GPT-3.5-turbo, and small samples. This paper tests whether the claim holds for newer models, other languages and expert readers, aiming to estimate the ceiling of human detection ability.

The authors built 16 datasets covering 9 languages and 9 domains, including community QA, news, tweets, Wikipedia-style text, student essays and peer reviews. For each, they sampled 300–600 human texts and generated counterparts with 11 SOTA LLMs, pairing a multilingual model with a language-specific one. 19 native-speaker NLP researchers labelled the texts under four task formats.

Contrary to the "random guessing" narrative, the experts reached 87.6% accuracy on average. The main tells were:

  • lack of concreteness,
  • weak cultural grounding,
  • low diversity.

Prompts that spelled out these differences helped in more than half of the cases. However, annotators did not reliably prefer human text, and they leaned toward machine output when unsure of its origin. The released data supports detection research and multilingual preference alignment.

Key points

  • Accuracy: across roughly 9K examples, expert accuracy ranged from 50% to 100% per setting, averaging 87.6%.

  • Task formats: single-binary, pair-binary, triplet-three-class and pair-four-class. Generators paired multilingual models (GPT/Claude series) with language-specific ones; for instance, Chinese was handled by ChatGLM or Qwen, while Arabic used AceGPT.

  • Observed gaps:

    • concreteness;
    • cultural nuance;
    • variety in length, structure, style and sentiment;
    • formatting;
    • code-mixing.

    Gap-aware prompting fully or partly fixed over 50% of cases, but cultural nuance and diversity remained hard to fix.

  • Preferences: these split roughly evenly between human and machine text, with a tilt toward MGT when the source was unclear.

  • Release (github.com/xnlp-lab/HumanEval-MGT):

    • 17k original and 32k improved MGTs;
    • 13.5k detection labels;
    • 5k preference judgments;
    • 1.6k fill-in-the-gap survey responses;
    • metadata on the 19 annotators.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cornell UniversityIthaca · United States120 works · 59,940 citations🎓Technische Universität DarmstadtDarmstadt · Germany109 works · 35,920 citations🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations🎓University of OsloOslo · Norway51 works · 4,155 citations🎓KU LeuvenLeuven · Belgium42 works · 5,839 citations🎓Alexandria UniversityAlexandria · Egypt39 works · 287 citations🔬Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo"Pisa · Italy4 works · 5,820 citations🎓Sofia University "St. Kliment Ohridski"Sofia · Bulgaria21 works · 2,595 citations🎓University of FloridaGainesville · United States33 works · 364 citations🎓Hanoi University of Science and TechnologyHanoi · Vietnam10 works · 89 citations🎓University of DelhiNew Delhi · India5 works · 15 citations🎓Trường Đại học Khoa học và Công nghệ Hà NộiHanoi · Vietnam8 works · 88 citations🎓Institute of Science TokyoTokyo · Japan10 works · 20 citations

Also on the byline: Zewail City of Science and Technology.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 works

Cite

Cite as Yuxia Wang, Rui Xing, Jonibek Mansurov, Giovanni Puccetti, Zhuohan Xie, Minh Ngoc Ta, Jiahui Geng, Jinyan Su, Mervat Abassy, Saadeldine Eletter, Kareem Elozeiri, Nurkhan Laiyk, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar, Alexander Aziz, Ryuto Koike, Masahiro Kaneko, Artem Shelmanov, Ekaterina Artemova, Vladislav Mikhailov, Akim Tsvigun, Alham Fikri Aji, Nizar Habash, Iryna Gurevych and Preslav Nakov. 2026. Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.639

@inproceedings{wang2026is,
  title = {{Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI}},
  author = {Yuxia Wang, Rui Xing, Jonibek Mansurov, Giovanni Puccetti, Zhuohan Xie, Minh Ngoc Ta, Jiahui Geng, Jinyan Su, Mervat Abassy, Saadeldine Eletter, Kareem Elozeiri, Nurkhan Laiyk, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar, Alexander Aziz, Ryuto Koike, Masahiro Kaneko, Artem Shelmanov, Ekaterina Artemova, Vladislav Mikhailov, Akim Tsvigun, Alham Fikri Aji, Nizar Habash, Iryna Gurevych and Preslav Nakov},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.639},
  url = {https://doi.org/10.18653/v1/2026.acl-long.639}
}

References

28 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Anthropic. 2024. The Claude 3 Model Family: Opus, Sonnet, Haiku. find
  2. Giovanni Bonisoli, Maria Pia di Buono, Laura Po, and Federica Rollo. 2023. DICE: A Dataset of Italian Crime Event news. In SIGIR ’23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, Taipei, Taiwan, July 23 - 27, 2023. ACM. find
  3. JM Chein, SA Martinez, and AR Barone. 2024. Human intelligence can safeguard against artificial intelligence: individual differences in the discernment of human from ai texts. Scientific Reports, 14(1):25989. find
  4. Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. 2021. All that‘s ‘human’ is not gold: Evaluating human evaluation of generated text. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International… find
  5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The Llama 3 Herd of Models. arXiv preprint arXiv:2407.21783.
  6. Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi, and Chris Callison-Burch. 2023. Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12763–1… find
  7. Omar Einea, Ashraf Elnagar, and Ridhwan Al Debsi. 2019. Sanad: Single-label Arabic news articles dataset for automatic text categorization. Data in Brief, 25:104076. find
  8. Cristina Garbacea, Samuel Carton, Shiyan Yan, and Qiaozhu Mei. 2019. Judge the judges: A large-scale evaluation study of neural language models for online review generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conf… find
  9. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is ChatGPT to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597.
  10. Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021. XL-sum: Large-scale multilingual abstractive summarization for 44 languages. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pa… find
  11. Jimpei Hitsuwari, Yoshiyuki Ueda, Woojin Yun, and Michio Nomura. 2023. Does human-AI collaboration lead to more creative art? aesthetic evaluation of human-made and ai-generated haiku poetry. Comput. Hum. Behav., 139:107502. find
  12. Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. 2020. Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1808–1822, Online. Association for Computational…
  13. Miao Li, Eduard Hovy, and Jey Han Lau. 2023. Summarizing multiple documents with conversational structure for meta-review generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7089–7112, Singapore. Association for Computational Linguistics. find
  14. Zaijing Li, Gongwei Chen, Rui Shao, Yuquan Xie, Dongmei Jiang, and Liqiang Nie. 2024. Enhancing emotional generation capability of large language models via emotional chain-of-thought. arXiv preprint arXiv:2401.06836. — [arXiv:2401.06836](https://arxiv.org/abs/2401.06836) find
  15. David M Markowitz, Jeffrey T Hancock, and Jeremy N Bailenson. 2024. Linguistic markers of inherently false AI communication and intentionally false human communication: Evidence from hotel reviews. Journal of Language and Social Psychology, 43(1):63–82. find
  16. Hannah Mieczkowski, Jeffrey T Hancock, Mor Naaman, Malte Jung, and Jess Hohenstein. 2021. AI-mediated communication: Language use and interpersonal effects in a referential communication task. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1):1–14. find
  17. MOP-LIWU Community and MNBVC Team. 2023. MNBVC: Massive Never-ending BT Vast Chinese corpus. https://github.com/esbatmop/MNBVC. ↗
  18. OpenAI. 2023. GPT-4 technical report. arXiv, abs/2303.08774.
  19. Marco Polignano, Pierpaolo Basile, and Giovanni Semeraro. 2024. Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA. find
  20. Tatiana Shamardina, Vladislav Mikhailov, Daniil Cher-nianskii, Alena Fenogenova, Marat Saidov, Anastasiya Valeeva, Tatiana Shavrina, Ivan Smurov, Elena Tutubalina, and Ekaterina Artemova. 2022. Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian. arXiv preprint arXiv:2… — [arXiv:2206.01583](https://arxiv.org/abs/2206.01583) find
  21. Tatiana Shamardina, Marat Saidov, Alena Fenogenova, Aleksandr Tumanov, Alina Zemlyakova, Anna Lebedeva, Ekaterina Gryaznova, Tatiana Shavrina, Vladislav Mikhailov, and Ekaterina Artemova. 2025. CoAT: Corpus of artificial texts. Natural Language Processing, 31(1):150–175. find
  22. Wei Song, Kai Zhang, Ruiji Fu, Lizhen Liu, Ting Liu, and Miaomiao Cheng. 2020. Multi-Stage Pre-training for Automated Chinese Essay Scoring. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6723–6733, Online. Association for Computational Lingu… find
  23. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  24. Chris van der Lee, Albert Gatt, Emiel van Miltenburg, Sander Wubben, and Emiel Krahmer. 2019. Best practices for the human evaluation of automatically generated text. In Proceedings of the 12th International Conference on Natural Language Generation, pages 355–368, Tokyo, Japan. Association for Comp… find
  25. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024a. M4GTbench: Evaluation benchmark for black-box machine-generated text detecti… find
  26. Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. 2024b. M4: Multi-generator, multi-domain, and multilingual black-box m… find
  27. Yuxia Wang, Artem Shelmanov, Jonibek Mansurov, Akim Tsvigun, Vladislav Mikhailov, Rui Xing, Zhuohan Xie, Jiahui Geng, Giovanni Puccetti, Ekaterina Artemova, Jinyan Su, Minh Ngoc Ta, Mervat Abassy, Kareem Ashraf Elozeiri, Saad El Dine Ahmed El Etter, Maiya Goloburda, Tarek Mahmoud, Raj Vardhan Tomar,… find
  28. Rustem Yeshpanov, Pavel Efimov, Leonid Boytsov, Ardak Shalkarbayuli, and Pavel Braslavski. 2024. KazQAD: Kazakh open-domain question answering dataset. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 964… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-is-human-like-text-liked/