Elamur Research⌕
☰
home / papers / 2026 / C-ReD: A Comprehensive Chinese Benchmark for AI-Generated Text Detection Derived from Real-World Prompts

C-ReD: A Comprehensive Chinese Benchmark for AI-Generated Text Detection Derived from Real-World Prompts

Paper · 2026 · Chenxi Qing, Junxi Wu, Zheng Liu et al.

Summary

Most existing benchmarks for detecting machine-written text are built on English. The few that cover Chinese tend to rely on ChatGPT alone, use mostly QA-style texts, and pair them with artificial prompts. C-ReD addresses these gaps with a Chinese benchmark.

It pairs human-written texts from five genres (news, Q&A, film reviews, exam essays, academic writing) with outputs from nine API-accessed LLMs, five of them Chinese domestic models. Generation prompts are modeled on realistic usage scenarios, and the outputs go through automated filtering and manual screening. A Traditional Chinese news subset provides an additional test setting.

The authors benchmark zero-shot detectors, supervised detectors, and LLMs used as judges. Results vary considerably by domain and generator, and Deepseek-R1's reasoning-style output is the hardest to catch. Fine-tuning on C-ReD sharply improves detection, and the gains carry over to unseen generators and to the external M4 dataset. This makes C-ReD a practical training and evaluation resource for Chinese AI-text detection.

Key points

  • Human-written sources:
    • 3,000 THUCNews articles
    • 2,956 Zhihu answers (Zhihu-KOL)
    • 2,960 Douban reviews (ChineseNlpCorpus)
    • 1,081 Gaokao model essays
    • 500 ChinaXiv papers
    • 2,000 Traditional Chinese news items (News-Collection-Zhtw)
  • Generators: gpt-3.5-turbo, gpt-4o, Gemini-2.5-Flash, Claude-3.5-Haiku, Deepseek-V3, Deepseek-R1 (chain-of-thought mode), Qwen2.5, Qwen3 and Doubao-1.5-Pro.
  • LLMs as judges: they perform poorly as plain zero-shot detectors, even on their own outputs, but improve when given domain context or style descriptions.
  • Academic writing: after C-ReD fine-tuning, RoBERTa-large's AUROC rises from about 0.37–0.40 to 0.97–0.99. Simplified prompts make generated text only marginally easier to detect.
  • Transfer to M4: training only on the Q&A domain (7 LLMs) raises AUROC on M4 ChatGPT texts from 0.6055 to 0.8169 for RoBERTa-base and from 0.5684 to 0.8890 for RoBERTa-large.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Harbin Institute of TechnologyHarbin · China158 works · 28,841 citations🔬Peng Cheng LaboratoryShenzhen · China38 works · 2,504 citations🎓Nankai UniversityTianjin · China24 works · 762 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as Chenxi Qing, Junxi Wu, Zheng Liu, Yixiang Qiu, Hongyao Yu, Bin Chen, Hao Wu and Shu-Tao Xia. 2026. C-ReD: A Comprehensive Chinese Benchmark for AI-Generated Text Detection Derived from Real-World Prompts. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.2119

@inproceedings{qing2026c,
  title = {{C-ReD: A Comprehensive Chinese Benchmark for AI-Generated Text Detection Derived from Real-World Prompts}},
  author = {Chenxi Qing, Junxi Wu, Zheng Liu, Yixiang Qiu, Hongyao Yu, Bin Chen, Hao Wu and Shu-Tao Xia},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2119},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2119}
}

References

25 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chow Aesop, Clauss Christian, Ding Li, and Zhu Su. 2023. Chinesenlpcorpus. https://github.com/ In-saneLife/ChineseNLPCorpus. ↗
  2. Sonnet Anthropic. 2024. Model card addendum: Claude 3.5 haiku and upgraded claude 3.5 sonnet. URL https://api.semanticscholar. org/CorpusID, 273639283. ↗
  3. Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2024. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations.
  4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  5. ByteDance. 2025. Doubao 1.5pro - doubao team. https://seed.bytedance.com/zh/special/doubao_1_5_pro. ↗
  6. Jiaqi Chen, Xiaoye Zhu, Tianyang Liu, Ying Chen, Chen Xinhui, Yiwen Yuan, Chak Tou Leong, Zuchao Li, Long Tang, Lei Zhang, and 1 others. 2025. Imitate before detect: Aligning machine stylistic preference for machine-revised text detection. In Proceedings of the AAAI Conference on Artificial Intellig… find
  7. ChinaXiv. 2025. Chinaxiv.org. https://chinaxiv.org. ↗
  8. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arX…
  9. Hao Fang, Jiawei Kong, Tianqu Zhuang, Yixiang Qiu, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Yaowei Wang, and Min Zhang. 2025. Your language model can secretly write like humans: Contrastive paraphrase attacks on llm-generated text detectors. In Proceedings of the 2025 Conference on Empirical Methods in N… find
  10. GeeYangML and Wang Ray. 2023. Zhihu-kol. https://github.com/wangrui6/Zhihu-KOL.42711 9 ↗
  11. Sebastian Gehrmann, Hendrik Strobelt, and Alexander M Rush. 2019. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111–116.
  12. Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597.
  13. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  14. Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2024. Mgtbench: Benchmarking machine-generated text detection. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 2251–2265. find
  15. Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. 2023. Radar: Robust ai-text detection via adversarial learning. Advances in neural information processing systems, 36:15077–15095. find
  16. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  17. Thomas Lavergne, Tanguy Urvoy, and François Yvon. 2008. Detecting fake content with relative entropy scoring. Pan, 8(27-31):4. find
  18. Hyunseok Lee, Jihoon Tack, and Jinwoo Shin. 2024. Remodetect: Reward models recognize aligned llm’s generations. Advances in Neural Information Processing Systems, 37:2886–2913. find
  19. Jooyoung Lee, Thai Le, Jinghui Chen, and Dongwon Lee. 2023. Do language models plagiarize? In Proceedings of the ACM Web Conference 2023, pages 3637–3647. find
  20. Jingyang Li, Maosong Sun, and Xian Zhang. 2006. A comparison and semi-quantitative analysis of words and character-bigrams as features in chinese text categorization. In proceedings of the 21st international conference on computational linguistics and 44th annual meeting of the association for compu… find
  21. Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2023. Mage: Machine-generated text detection in the wild. arXiv preprint arXiv:2305.13242.
  22. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  23. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  24. Dominik Macko, Robert Moro, Adaku Uchendu, Jason Samuel Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, and 1 others. 2023. Multitude: Large-scale multilingual machine-generated text detection benchmark. arXiv preprint arXiv:2310.13606. find
  25. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning, pages 24950–24962. PMLR.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/qing-2026-c-red-comprehensive-chinese-benchmark/