Elamur Research⌕
☰
home / papers / 2025 / 中文对抗攻击下的ChatGPT鲁棒性评估

中文对抗攻击下的ChatGPT鲁棒性评估

Summary

Large language models such as ChatGPT are widely deployed, but little work has measured how they hold up against adversarial text, especially in Chinese. Most earlier attack studies targeted CNNs, RNNs or Transformer-based pre-trained models. The authors, from Harbin Institute of Technology and Antiy Labs, define a new quantity called offset average difference (OAD) and use it to build a numeric robustness metric, the OAD-based robustness score (ORS).

In a black-box setting, they generate adversarial examples with nine common word-importance-based Chinese attack methods, some of them adapted from English attacks. Each method's attack success rate against the model is turned into a per-method score and then averaged. The basic ORS works for models that return only hard labels, like ChatGPT. A soft-label variant also counts how many adversarial texts are misclassified with high confidence.

The same scoring idea is reused as OFS, which has ChatGPT rate adversarial-text fluency automatically instead of human annotators. On Chinese news and sentiment classification, ChatGPT scores about 20% higher than Chinese BERT. Some attacks still fool it more than 40% of the time.

Key points

  • New concepts and metrics:
    • OAD (offset average difference) is the underlying quantity.
    • ORS measures model robustness and comes in hard-label and soft-label versions; the soft-label version adds the share of high-confidence misclassifications.
    • OFS is an automated fluency score that uses ChatGPT as the judge.
  • Attack setup: black-box evasion attacks using nine Chinese adversarial generation methods built on greedy word-importance ranking and perturbation. The pool includes established Chinese methods (e.g., pinyin-homophone, glyph and character-splitting substitutions) plus methods transferred from English.
  • Target models and data:
    • The models attacked are ChatGPT and Chinese BERT.
    • The tasks are real-world Chinese news classification and sentiment classification. The reference list cites THUCTC and ChineseNlpCorpus, which are likely the data sources.
  • Results:
    • ChatGPT's average ORS is roughly 20% higher than Chinese BERT's.
    • The strongest attack still reaches a success rate above 40% against ChatGPT.
    • The authors describe these findings as preliminary.
  • Publication: Zhang Yunting, Ye Lin, Li Baisong, Zhang Hongli, Journal of Software (软件学报) 36(10):4710–4734, 2025, written in Chinese, doi:10.13328/j.cnki.jos.007299.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.13328/j.cnki.jos.007299

@article{2025,
  title = {{中文对抗攻击下的ChatGPT鲁棒性评估}},
  author = {张云婷 and 叶麟 and 李柏松 and 张宏莉},
  journal = {软件学报 (Journal of Software)},
  year = {2025},
  doi = {10.13328/j.cnki.jos.007299},
  url = {https://doi.org/10.13328/j.cnki.jos.007299}
}

References

34 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R. Intriguing properties of neural networks. In: Proc. of the 2014 Int’l Conf. on Learning Representations. OpenReview.net. 2014. find our card
  2. Goodfellow IJ, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. arXiv:1412.6572, 2015. find our card
  3. Gao J, Lanchantin J, Soffa ML, Qi YJ. Black-box generation of adversarial text sequences to evade deep learning classifiers. In: Proc. of the 2018 IEEE Security and Privacy Workshops. San Francisco: IEEE, 2018. 50–56. [doi: 10.1109/SPW.2018.00016] find our card
  4. Wang WQ, Wang R, Wang LN, Tang BX. Adversarial examples generation approach for tendency classification on Chinese texts. Ruan Jian Xue Bao/Journal of Software, 2019, 30(8): 2415–2427 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/5765. htm [doi: 10.13328/j.cnki.jos.005765] — [doi:10.13328/j.cnki.jos.005765](https://doi.org/10.13328/j.cnki.jos.005765) ↗
  5. Zhu KJ, Wang JD, Zhou JH, Wang ZC, Chen H, Wang YD, Yang LY, Ye W, Zhang Y, Gong N, Xie X. PromptRobust: Towards evaluating the robustness of large language models on adversarial prompts. arXiv:2306.04528, 2024. find our card
  6. Liang B, Li HC, Su MQ, Bian P, Li XR, Shi WC. Deep text classification can be fooled. In: Proc. of the 27th Int’l Joint Conf. on Artificial Intelligence. Stockholm: AAAI Press, 2018. 4208–4215. find
  7. Wang WQ, Wang R, Wang LN, Wang ZB, Ye AS. Towards a robust deep neural network in texts: A survey. arXiv:1902.07285, 2021. fs 0.40 0.00 0.00 0.00 0.55 0.00 0.11 0.00 0.11 0.22 10.00 10.00 10.00 9.96 10.00 9.98 10.00 9.99 9.94 9.99 — [arXiv:1902.07285](https://arxiv.org/abs/1902.07285) find
  8. Alzantot M, Sharma Y, Elgohary A, Ho BJ, Srivastava M, Chang KW. Generating natural language adversarial examples. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Brussels: Association for Computational Linguistics, 2018. 2890–2896. [doi: 10.18653/v1/D18-1316] find our card
  9. Zang Y, Qi FC, Yang CH, Liu ZY, Zhang M, Liu Q, Sun MS. Word-level textual adversarial attacking as combinatorial optimization. In: Proc. of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020. 6066–6080. [doi: 10.18653/v1/2020.ac… find our card
  10. Jin D, Jin ZJ, Zhou JT, Szolovits P. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI Press, 2020. 8018–8025. [doi: 10.1609/aaai. v34i05.6311] find our card
  11. Li LY, Ma RT, Guo QP, Xue XY, Qiu XP. BERT-ATTACK: Adversarial attack against BERT using BERT. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2020. 6193–6202. [doi: 10.18653/v1/2020.emnlp-main.500] find our card
  12. Ren SH, Deng YH, He K, Che WX. Generating natural language adversarial examples through probability weighted word saliency. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: Association for Computational Linguistics, 2019. 1085–1097. [doi: 10.18653/v1/P… find our card
  13. Xu JC, Du QF. Adversarial attacks on text classification models using layer-wise relevance propagation. Int’l Journal of Intelligent Systems, 2020, 35(9): 1397–1415. [doi: 10.1002/int.22260] — [doi:10.1002/int.22260](https://doi.org/10.1002/int.22260) find
  14. Li JF, Ji SL, Du TY, Li B, Wang T. TextBugger: Generating adversarial text against real-world applications. In: Proc. of the 26th Annual Network and Distributed System Security Symp. San Diego: The Internet Society, 2019. [doi: 10.14722/ndss.2019.23138] find our card
  15. Garg S, Ramakrishnan G. BAE: BERT-based adversarial examples for text classification. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2020. 6174–6181. [doi: 10.18653/v1/2020.emnlp-main.498] find our card
  16. Li DQ, Zhang YZ, Peng H, Chen LQ, Brockett C, Sun MT, Dolan B. Contextualized perturbation for textual adversarial attack. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistic… find
  17. Zhang ZH, Liu MX, Zhang C, Zhang YM, Li Z, Li Q, Duan HX, Sun DH. Argot: Generating adversarial readable Chinese texts. In: Proc. of the 29th Int’l Joint Conf. on Artificial Intelligence. Yokohama, 2021. 2533–2539. [doi: 10.24963/ijcai.2020/351] — [doi:10.24963/ijcai.2020/351](https://doi.org/10.24963/ijcai.2020/351) find
  18. Cheng N, Chang GQ, Gao HC, Pei G, Zhang Y. WordChange: Adversarial examples generation approach for Chinese text classification. IEEE Access, 2020, 8: 79561–79572. [doi: 10.1109/ACCESS.2020.2988786] — [doi:10.1109/access.2020.2988786](https://doi.org/10.1109/access.2020.2988786) find
  19. Tong X, Wang LN, Wang RZ, Wang JY. A generation method of word-level adversarial samples for Chinese text classification. Netinfo Security, 2020, 20(9): 12–16 (in Chinese with English abstract). [doi: 10.3969/j.issn.1671-1122.2020.09.003] — [doi:10.3969/j.issn.1671-1122.2020.09.003](https://doi.org/10.3969/j.issn.1671-1122.2020.09.003) find
  20. Ou HX, Yu L, Tian SW, Chen X. Chinese adversarial examples generation approach with multi-strategy based on semantic. Knowledge and Information Systems, 2022, 64(4): 1101–1119. [doi: 10.1007/s10115-022-01652-1] — [doi:10.1007/s10115-022-01652-1](https://doi.org/10.1007/s10115-022-01652-1) find
  21. Zhang YT, Ye L, Tang HL, Zhang HL, Li S. Chinese BERT attack method based on masked language model. Ruan Jian Xue Bao/Journal of Software, 2024, 35(7): 3392–3409 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6932.htm [doi: 10.13328/j.cnki. jos.006932] — [doi:10.13328/j.cnki](https://doi.org/10.13328/j.cnki) ↗
  22. He XL, Lyu LJ, Sun LC, Xu QK. Model extraction and adversarial transferability, your BERT is vulnerable! In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2021. 2006–2012… find
  23. Ebrahimi J, Rao AY, Lowd D, Dou DJ. HotFlip: White-box adversarial examples for text classification. In: Proc. of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 2: Short Papers. Melbourne: Association for Computational Linguistics, 2018. 31–36. [doi: 10.18653/v1/P18-2… find our card
  24. Shi YC, Han YH. Metric system and its completeness of adversarial robustness evaluation. Ruan Jian Xue Bao/Journal of Software, 2025, 36(3): 1304–1326 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/7172.htm [doi: 10.13328/j.cnki.jos.007172] — [doi:10.13328/j.cnki.jos.007172](https://doi.org/10.13328/j.cnki.jos.007172) ↗
  25. Yoo JY, Morris JX, Lifland E, Qi YJ. Searching for a search method: Benchmarking search algorithms for generating NLP adversarial examples. In: Proc. of the 3rd BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP. Association for Computational Linguistics, 2020. 323–332. [doi:… find
  26. Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional Transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1: Long and Short Papers. Minneapoli… find
  27. Mrkšić N, Séaghdha DÓ, Thomson B, Gašić M, Rojas-Barahona LM, Su PH, Vandyke D, Wen TH, Young S. Counter-fitting word vectors to linguistic constraints. In: Proc. of the 2016 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego:… find
  28. Gu JJ, Li XS. The effects of character transposition within and across words in Chinese reading. Attention, Perception, & Psychophysics, 2015, 77(1): 272–281. [doi: 10.3758/s13414-014-0749-5] — [doi:10.3758/s13414-014-0749-5](https://doi.org/10.3758/s13414-014-0749-5) find
  29. Xu LH, Lin HF, Pan Y, Ren H, Chen JM. Constructing the affective lexicon ontology. Journal of the China Society for Scientific and Technical Information, 2008, 27(2): 180–185 (in Chinese with English abstract). [doi: 10.3969/j.issn.1000-0135.2008.02.004] — [doi:10.3969/j.issn.1000-0135.2008.02.004](https://doi.org/10.3969/j.issn.1000-0135.2008.02.004) find
  30. Sun MS, Li JY, Guo ZP, Zhao Y, Zheng YB, Si XC, Liu ZY. THUCTC: An efficient Chinese text classifier. Technnical Report, Beijing: Tsinghua University, 2016 (in Chinese with English abstract). http://thuctc.thunlp.org/ ↗
  31. ChineseNlpCorpus. GitHub, 2018. https://github.com/SophonPlus/ChineseNlpCorpus ↗
  32. Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv:1412.6980, 2017. find
  33. Kusner MJ, Sun Y, Kolkin NI, Weinberger KQ. From word embeddings to document distances. In: Proc. of the 32nd Int’l Conf. on Machine Learning. Lille: JMLR.org, 2015. 957–966. find
  34. Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv:1301.3781, 2013. 附中文参考文献: [4] 王文琦, 汪润, 王丽娜, 唐奔宵. 面向中文文本倾向性分类的对抗样本生成方法. 软件学报, 2019, 30(8): 2415–2427. http://www.jos.org.cn/1000-9825/5765.htm [doi: 10.13328/j.cnki.jos.005765] [19] 仝鑫, 王罗娜, 王润正… ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2025-robustness-evaluation-chatgpt/