Elamur Research
☰
home / papers / 2023 / A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly

A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2312.02003

Summary

This survey maps how large language models relate to cybersecurity and privacy from three directions. The authors reviewed 281 publications and grouped them into helpful uses of LLMs for defenders ("The Good"), offensive uses by attackers ("The Bad"), and weaknesses of the models themselves along with countermeasures ("The Ugly"). Defensive work covers the full code life cycle, from secure coding and test generation to vulnerability and malware detection and repair. It also covers data protection: integrity, confidentiality, reliability and traceability. Most studies report that LLM-based approaches beat conventional techniques. Attacks are sorted by level: hardware, OS, software, network and user. User-level misuse, such as misinformation and social engineering, is the largest group because LLMs reason in a human-like way. Model vulnerabilities are split into AI-inherent and non-AI-inherent types. Defenses act at the architecture, training or inference stage. The survey matters because it is among the first to cover all three perspectives together. It also identifies gaps that need research, including model and parameter extraction and safe instruction tuning.

Key points

  • The corpus contains 281 papers. Beneficial security applications form the largest share, which the authors read as LLMs helping security more than harming it.
  • A motivating example: GPT-3 reportedly found 213 vulnerabilities in a repository, only 4 of them false positives. A leading commercial tool found 99.
  • Offensive uses are classed into five levels: hardware, OS, software, network and user. User-level attacks lead with 32 papers.
  • Vulnerabilities are divided into two groups:
    • AI-inherent: data poisoning, backdoors, training-data extraction.
    • Non-AI-inherent: remote code execution, prompt injection, side channels.
  • Defenses are sorted by where they act:
    • Training stage: corpus cleaning, optimization methods.
    • Inference stage: instruction pre-processing, malicious-input detection, output post-processing.
  • Open gaps:
    • Model and parameter extraction work remains mostly theoretical.
    • The effect of model architecture on safety is little studied.
    • Safe instruction tuning needs more exploration.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

122 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 121 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1016/j.hcc.2024.100211 · arXiv 2312.02003

@misc{yao2023survey,
  title = {{A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly}},
  author = {Yao, Yifan and Duan, Jinhao and Xu, Kaidi and Cai, Yuanfang and Sun, Zhibo and Zhang, Yue},
  year = {2023},
  doi = {10.1016/j.hcc.2024.100211},
  eprint = {2312.02003},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1016/j.hcc.2024.100211}
}

References

353 references, 38 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Abbasian, I. Azimi, A. M. Rahmani, and R. Jain, “Conversational health agents: A personalized llm-powered agent framework,” 2023. find
  2. H. Aghakhani, W. Dai, A. Manoel, X. Fernandes, A. Kharkar, C. Kruegel, G. Vigna, D. Evans, B. Zorn, and R. Sim, “Trojanpuzzle: Covertly poisoning code-suggestion models,” arXiv preprint arXiv:2301.02344, 2023. — [arXiv:2301.02344](https://arxiv.org/abs/2301.02344) find
  3. B. Ahmad, S. Thakur, B. Tan, R. Karri, and H. Pearce, “Fixing hardware security bugs with large language models,” arXiv preprint arXiv:2302.01215, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2302.01215 — [doi:10.48550/arxiv.2302.01215](https://doi.org/10.48550/arxiv.2302.01215) ↗
  4. M. AI, “Introducing llama: A foundational, 65-billion-parameter language model,” https://ai.meta.com/blog/ large-language-model-llama-meta-ai/, feb 2023, accessed: 2023-11-13. ↗
  5. M. Al-Hawawreh, A. Aljuhani, and Y. Jararweh, “Chatgpt for cybersecurity: practical applications, challenges, and future directions,” Cluster Computing, vol. 26, no. 6, pp. 3421–3436, 2023. find
  6. S. Alagarsamy, C. Tantithamthavorn, and A. Aleti, “A3test: Assertion-augmented automated test case generation,” arXiv preprint arXiv:2302.10352, 2023. — [arXiv:2302.10352](https://arxiv.org/abs/2302.10352) find
  7. M. Alawida, B. A. Shawar, O. I. Abiodun, A. Mehmood, A. E. Omolara et al., “Unveiling the dark side of chatgpt: Exploring cyberattacks and enhancing user awareness,” 2023. find
  8. T. Ali and P. Kostakos, “Huntgpt: Integrating machine learning-based anomaly detection and explainable ai with large language models (llms),” arXiv preprint arXiv:2309.16021, 2023. — [arXiv:2309.16021](https://arxiv.org/abs/2309.16021) find
  9. E. Ambikairajah, H. Li, L. Wang, B. Yin, and V. Sethu, “Language identification: A tutorial,” IEEE Circuits and Systems Magazine, vol. 11, no. 2, pp. 82–108, 2011. find
  10. Z. Amos, “What is fraudgpt?” https://hackernoon.com/ what-is-fraudgpt, 2023. ↗
  11. J. R. Anderson and C. J. Lebiere, The atomic components of thought. Psychology Press, 2014. find
  12. Anonymous, “On the safety of open-sourced large language models: Does alignment really prevent them from being misused?” in Submitted to The Twelfth International Conference on Learning Representations, 2023, under review. [Online]. Available: https://openreview.net/forum?id=E6Ix4ahpzd ↗
  13. B. B. Arcila, “Is it a platform? is it a search engine? it’s chatgpt! the european liability regime for large language models,” J. Free Speech L., vol. 3, p. 455, 2023. find
  14. A. H. Bailey, A. Williams, and A. Cimpian, “Based on billions of words on the internet, people= men,” Science Advances, vol. 8, no. 13, p. eabm2463, 2022. find
  15. A. Bakhshandeh, A. Keramatfar, A. Norouzi, and M. M. Chekidehkhoun, “Using chatgpt as a static application security testing tool,” arXiv preprint arXiv:2308.14434, 2023. — [arXiv:2308.14434](https://arxiv.org/abs/2308.14434) find
  16. S. Barikeri, A. Lauscher, I. Vulić, and G. Glavaš, “Redditbias: A real-world resource for bias evaluation and debiasing of conversational language models,” arXiv preprint arXiv:2106.03521, 2021. find
  17. M. Beckerich, L. Plein, and S. Coronado, “Ratgpt: Turning online llms into proxies for malware attacks,” 2023. find
  18. S. Ben-Moshe, G. Gekker, and G. Cohen, “Opwnai: Ai that can save the day or hack it away. check point research (2022),” 2023. find
  19. A.-R. Bhojani and M. Schwarting, “Truth and regret: Large language models, the quran, and misinformation,” pp. 1–7, 2023. find
  20. F. Bianchi, M. Suzgun, G. Attanasio, P. Röttger, D. Jurafsky, T. Hashimoto, and J. Zou, “Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions,” arXiv preprint arXiv:2309.07875, 2023. find our card
  21. S. Bordia and S. R. Bowman, “Identifying and reducing gender bias in word-level language models,” arXiv preprint arXiv:1904.03035, 2019. find our card
  22. M. Botacin, “Gpthreats-3: Is automatic malware generation a threat?” in 2023 IEEE Security and Privacy Workshops (SPW). IEEE, 2023, pp. 238–254. find
  23. S. R. Bowman, “Eight things to know about large language models,” arXiv preprint arXiv:2304.00612, 2023. — [arXiv:2304.00612](https://arxiv.org/abs/2304.00612) find
  24. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… find
  25. M. Burgess, “Chatgpt has a plug-in problem,” https://www.wired. com/story/chatgpt-plugins-security-privacy-risk/, 2023. ↗
  26. Y. Cai, S. Mao, W. Wu, Z. Wang, Y. Liang, T. Ge, C. Wu, W. You, T. Song, Y. Xia et al., “Low-code llm: Visual programming over llms,” arXiv preprint arXiv:2304.08103, 2023. — [arXiv:2304.08103](https://arxiv.org/abs/2304.08103) find
  27. B. Cao, Y. Cao, L. Lin, and J. Chen, “Defending against alignment-breaking attacks via robustly aligned llm,” 2023. find
  28. N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022, pp. 1897– 1914. find
  29. N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., “Extracting training data from large language models,” in 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 2633–2650. find our card
  30. P. Caven, “A more insecure ecosystem? chatgpt’s influence on cybersecurity,” ChatGPT’s Influence on Cybersecurity (April 30, 2023), 2023. find
  31. Y. Chang, X. Wang, J. Wang, Y. Wu, K. Zhu, H. Chen, L. Yang, X. Yi, C. Wang, Y. Wang et al., “A survey on evaluation of large language models,” arXiv preprint arXiv:2307.03109, 2023. find our card
  32. P. V. S. Charan, H. Chunduri, P. M. Anand, and S. K. Shukla, “From text to mitre techniques: Exploring the malicious use of large language models for generating cyber attack payloads,” 2023. find
  33. B. Chen, F. Zhang, A. Nguyen, D. Zan, Z. Lin, J.-G. Lou, and W. Chen, “Codet: Code generation with generated tests,” arXiv preprint arXiv:2207.10397, 2022. find
  34. B. Chen, A. Paliwal, and Q. Yan, “Jailbreaker in jail: Moving target defense for large language models,” arXiv preprint arXiv:2310.02417, 2023. — [arXiv:2310.02417](https://arxiv.org/abs/2310.02417) find
  35. C. Chen and K. Shu, “Can llm-generated misinformation be detected?” 2023. find
  36. ——, “Combating misinformation in the age of llms: Opportunities and challenges,” arXiv preprint arXiv:2311.05656, 2023. find
  37. C. Chen, J. Su, J. Chen, Y. Wang, T. Bi, Y. Wang, X. Lin, T. Chen, and Z. Zheng, “When chatgpt meets smart contract vulnerability detection: How far are we?” arXiv preprint arXiv:2309.05520, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2309.05520 — [doi:10.48550/arxiv.2309.05520](https://doi.org/10.48550/arxiv.2309.05520) ↗
  38. T. Chen, L. Li, L. Zhu, Z. Li, G. Liang, D. Li, Q. Wang, and T. Xie, “Vullibgen: Identifying vulnerable third-party libraries via generative pre-trained model,” arXiv preprint arXiv:2308.04662, 2023. [Online]. Available: https: //doi.org/10.48550/arXiv.2308.04662 — [doi:10.48550/arxiv.2308.04662](https://doi.org/10.48550/arxiv.2308.04662) find
  39. X. Chen, S. Jia, and Y. Xiang, “A review: Knowledge reasoning over knowledge graph,” Expert Systems with Applications, vol. 141, p. 112948, 2020. find
  40. Y. Chen, A. Arunasalam, and Z. B. Celik, “Can large language models provide security & privacy advice? measuring the ability of llms to refute misconceptions,” 2023. find
  41. A. Cheshkov, P. Zadorozhny, and R. Levichev, “Evaluation of chatgpt model for vulnerability detection,” arXiv preprint arXiv:2304.07232, 2023. [Online]. Available: https://doi.org/10. 48550/arXiv.2304.07232 — [arXiv:2304.07232](https://arxiv.org/abs/2304.07232) find
  42. C. A. Choquette-Choo, F. Tramer, N. Carlini, and N. Papernot, “Label-only membership inference attacks,” in International conference on machine learning. PMLR, 2021, pp. 1964–1974. find
  43. M. Chowdhury, N. Rifat, S. Latif, M. Ahsan, M. S. Rahman, and R. Gomes, “Chatgpt: The curious case of attack vectors’ supply chain management improvement,” in 2023 IEEE International Conference on Electro Information Technology (eIT), 2023, pp. 499–504. find
  44. J. Clusmann, F. R. Kolbinger, H. S. Muti, Z. I. Carrero, J.-N. Eckardt, N. G. Laleh, C. M. L. Löffler, S.-C. Schwarzkopf, M. Unger, G. P. Veldhuizen et al., “The future landscape of large language models in medicine,” Communications Medicine, vol. 3, no. 1, p. 141, 2023. find
  45. D. R. Cotton, P. A. Cotton, and J. R. Shipway, “Chatting and cheating: Ensuring academic integrity in the era of chatgpt,” Innovations in Education and Teaching International, pp. 1–12, 2023. find
  46. G. M. Currie, “Academic integrity and artificial intelligence: is chatgpt hype, hero or heresy?” in Seminars in Nuclear Medicine. Elsevier, 2023. find
  47. S. Dai, Y. Zhou, L. Pang, W. Liu, X. Hu, Y. Liu, X. Zhang, and J. Xu, “Llms may dominate information access: Neural retrievers are biased towards llm-generated texts,” arXiv preprint arXiv:2310.20501, 2023. — [arXiv:2310.20501](https://arxiv.org/abs/2310.20501) find
  48. D. Dale, A. Voronov, D. Dementieva, V. Logacheva, O. Kozlova, N. Semenov, and A. Panchenko, “Text detoxification using large pre-trained neural models,” arXiv preprint arXiv:2109.08914, 2021. find
  49. B. Dash and P. Sharma, “Are chatgpt and deepfake algorithms endangering the cybersecurity industry? a review,” International Journal of Engineering and Applied Sciences, vol. 10, no. 1, 2023. find
  50. Databricks, “Free dolly: Introducing the world’s first open and commercially viable instruction-tuned llm,” https://www.databricks.com/blog/2023/04/12/ dolly-first-open-commercially-viable-instruction-tuned-llm, 2023, accessed: 2023-11-13. ↗
  51. E. Debenedetti, G. Severi, N. Carlini, C. A. Choquette-Choo, M. Jagielski, M. Nasr, E. Wallace, and F. Tramèr, “Privacy side channels in machine learning systems,” arXiv preprint arXiv:2309.05610, 2023. — [arXiv:2309.05610](https://arxiv.org/abs/2309.05610) find
  52. D. Delley, “Wormgpt – the generative ai tool cybercriminals are using to launch business email compromise attacks,” https://shorturl. at/iwFL7, 2023. find
  53. G. Deng, Y. Liu, Y. Li, K. Wang, Y. Zhang, Z. Li, H. Wang, T. Zhang, and Y. Liu, “Jailbreaker: Automated jailbreak across multiple large language model chatbots,” arXiv preprint arXiv:2307.08715, 2023. — [arXiv:2307.08715](https://arxiv.org/abs/2307.08715) find
  54. ——, “Masterkey: Automated jailbreaking of large language model chatbots,” in Proceedings of the 31th Annual Network and Distributed System Security Symposium (NDSS’24), 2024. find
  55. G. Deng, Y. Liu, V. Mayoral-Vilches, P. Liu, Y. Li, Y. Xu, T. Zhang, Y. Liu, M. Pinzger, and S. Rass, “Pentestgpt: An llm-empowered automatic penetration testing tool,” arXiv preprint arXiv:2308.06782, 2023. — [arXiv:2308.06782](https://arxiv.org/abs/2308.06782) find
  56. Y. Deng, C. S. Xia, H. Peng, C. Yang, and L. Zhang, “Fuzzing deep-learning libraries via large language models,” arXiv preprint arXiv:2212.14834, 2022. — [arXiv:2212.14834](https://arxiv.org/abs/2212.14834) find
  57. Y. Deng, C. S. Xia, C. Yang, S. D. Zhang, S. Yang, and L. Zhang, “Large language models are edge-case fuzzers: Testing deep learning libraries via fuzzgpt,” arXiv preprint arXiv:2304.02014, 2023. — [arXiv:2304.02014](https://arxiv.org/abs/2304.02014) find
  58. ——, “Large language models are edge-case generators: Crafting unusual programs for fuzzing deep learning libraries,” in 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE), 2024, pp. 830–842. find
  59. E. Derner, K. Batistič, J. Zahálka, and R. Babuška, “A security risk taxonomy for large language models,” arXiv preprint arXiv:2311.11415, 2023. — [arXiv:2311.11415](https://arxiv.org/abs/2311.11415) find
  60. E. Derner and K. Batistič, “Beyond the safeguards: Exploring the security risks of chatgpt,” 2023. find
  61. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2019. find
  62. P. Dhoni and R. Kumar, “Synergizing generative ai and cybersecurity: Roles of generative ai entities, companies, agencies, and government in enhancing cybersecurity,” 2023. find
  63. H. Ding, V. Kumar, Y. Tian, Z. Wang, R. Kwiatkowski, X. Li, M. K. Ramanathan, B. Ray, P. Bhatia, S. Sengupta et al., “A static evaluation of code completion by large language models,” arXiv preprint arXiv:2306.03203, 2023. find
  64. X. Ding, L. Chen, M. Emani, C. Liao, P.-H. Lin, T. Vanderbruggen, Z. Xie, A. Cerpa, and W. Du, “Hpc-gpt: Integrating large language model for high-performance computing,” in Proceedings of the SC ’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analy… — [doi:10.1145/3624062.3624172](https://doi.org/10.1145/3624062.3624172) find
  65. X. Dong, Y. Wang, P. S. Yu, and J. Caverlee, “Probing explicit and implicit gender bias through llm conditional text generation,” arXiv preprint arXiv:2311.00306, 2023. — [arXiv:2311.00306](https://arxiv.org/abs/2311.00306) find
  66. X. Dong, A. T. Luu, M. Lin, S. Yan, and H. Zhang, “How should pre-trained language models be fine-tuned towards adversarial robustness?” Advances in Neural Information Processing Systems, vol. 34, pp. 4356–4369, 2021. find
  67. J. Duan, H. Cheng, S. Wang, C. Wang, A. Zavalny, R. Xu, B. Kailkhura, and K. Xu, “Shifting attention to relevance: Towards the uncertainty estimation of large language models,” arXiv preprint arXiv:2307.01379, 2023. find
  68. J. Duan, F. Kong, S. Wang, X. Shi, and K. Xu, “Are diffusion models vulnerable to membership inference attacks?” in Proceedings of the 40th International Conference on Machine Learning, 2023, pp. 8717–8730. find
  69. F. Duarte, “Number of chatgpt users (nov 2023),” https:// explodingtopics.com/blog/chatgpt-users, 2023, accessed: 2023-11-13. find
  70. C. Dwork, “Differential privacy,” in International colloquium on automata, languages, and programming. Springer, 2006, pp. 1–12. find
  71. C. Egersdoerfer, D. Zhang, and D. Dai, “Early exploration of using chatgpt for log-based anomaly detection on parallel file systems logs,” 2023. find
  72. D. O. Eke, “Chatgpt and the rise of generative ai: threat to academic integrity?” Journal of Responsible Technology, vol. 13, p. 100060, 2023. find
  73. A. Elhafsi, R. Sinha, C. Agia, E. Schmerling, I. A. Nesnas, and M. Pavone, “Semantic anomaly detection with large language models,” Autonomous Robots, pp. 1–21, 2023. find
  74. S. Eli and D. Gil, “Self-enhancing pattern detection with llms: Our answer to uncovering malicious packages at scale,” https: //apiiro.com/blog/llm-code-pattern-malicious-package-detection/, 2023, accessed: 2023-11-13. find
  75. T. Espinha Gasiba, K. Oguzhan, I. Kessba, U. Lechner, and M. Pinto-Albuquerque, “I’m sorry dave, i’m afraid i can’t fix your code: On chatgpt, cybersecurity, and secure coding,” in 4th International Computer Programming Education Conference (ICPEC 2023). Schloss-Dagstuhl-Leibniz Zentrum für Informat… find
  76. P. V. Falade, “Decoding the threat landscape: Chatgpt, fraudgpt, and wormgpt in social engineering attacks,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, p. 185–198, Oct. 2023. [Online]. Available: http://dx.doi.org/10.32628/CSEIT2390533 — [doi:10.32628/cseit2390533](https://doi.org/10.32628/cseit2390533) ↗
  77. A. Fan, B. Gokkaya, M. Harman, M. Lyubarskiy, S. Sengupta, S. Yoo, and J. M. Zhang, “Large language models for software engineering: Survey and open problems,” 2023. find
  78. T. Fan, Y. Kang, G. Ma, W. Chen, W. Wei, L. Fan, and Q. Yang, “Fate-llm: A industrial grade federated learning framework for large language models,” arXiv preprint arXiv:2310.10049, 2023. — [arXiv:2310.10049](https://arxiv.org/abs/2310.10049) find
  79. X. Fang, S. Che, M. Mao, H. Zhang, M. Zhao, and X. Zhao, “Bias of ai-generated content: An examination of news produced by large language models,” arXiv preprint arXiv:2309.09825, 2023. — [arXiv:2309.09825](https://arxiv.org/abs/2309.09825) find
  80. J. C. Farah, B. Spaenlehauer, V. Sharma, M. J. Rodríguez-Triana, S. Ingram, and D. Gillet, “Impersonating chatbots in a code review exercise to teach software engineering best practices,” in 2022 IEEE Global Engineering Education Conference (EDUCON). IEEE, 2022, pp. 1634–1642. find
  81. V. K. Felkner, H.-C. H. Chang, E. Jang, and J. May, “Winoqueer: A community-in-the-loop benchmark for anti-lgbtq+ bias in large language models,” arXiv preprint arXiv:2306.15087, 2023. find
  82. S. Y. Feng, V. Gangal, J. Wei, S. Chandar, S. Vosoughi, T. Mitamura, and E. Hovy, “A survey of data augmentation approaches for nlp,” arXiv preprint arXiv:2105.03075, 2021. find our card
  83. M. Fu, C. Tantithamthavorn, V. Nguyen, and T. Le, “Chatgpt for vulnerability detection, classification, and repair: How far are we?” 2023. find
  84. W. Fu, H. Wang, C. Gao, G. Liu, Y. Li, and T. Jiang, “Practical membership inference attacks against fine-tuned large language models via self-prompt calibration,” 2023. find
  85. ——, “A probabilistic fluctuation based membership inference attack for diffusion models,” 2023. find
  86. P. Ganesh, H. Chang, M. Strobel, and R. Shokri, “On the impact of machine learning randomness on group fairness,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 2023, pp. 1789–1800. find
  87. C. A. Gao, F. M. Howard, N. S. Markov, E. C. Dyer, S. Ramesh, Y. Luo, and A. T. Pearson, “Comparing scientific abstracts generated by chatgpt to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers,” BioRxiv, pp. 2022–12, 2022. find
  88. S. Gehman, S. Gururangan, M. Sap, Y. Choi, and N. A. Smith, “Realtoxicityprompts: Evaluating neural toxic degeneration in language models,” arXiv preprint arXiv:2009.11462, 2020. find our card
  89. K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “More than you’ve asked for: A comprehensive analysis of novel prompt injection threats to application-integrated large language models,” arXiv preprint arXiv:2302.12173, 2023. find our card
  90. Q. Gu, “Llm-based code generation method for golang compiler testing,” 2023. find
  91. Z. Gu, B. Zhu, G. Zhu, Y. Chen, M. Tang, and J. Wang, “Anomalygpt: Detecting industrial anomalies using large vision-language models,” arXiv preprint arXiv:2308.15366, 2023. — [arXiv:2308.15366](https://arxiv.org/abs/2308.15366) find
  92. M. Gupta, C. Akiri, K. Aryal, E. Parker, and L. Praharaj, “From chatgpt to threatgpt: Impact of generative ai in cybersecurity and privacy,” IEEE Access, 2023. find
  93. M. U. Hadi, R. Qureshi, A. Shah, M. Irfan, A. Zafar, M. Shaikh, N. Akhtar, J. Wu, and S. Mirjalili, “A survey on large language models: Applications, challenges, limitations, and practical usage,” TechRxiv, 2023. find
  94. A. Happe and J. Cito, “Getting pwn’d by ai: Penetration testing with large language models,” arXiv preprint arXiv:2308.00121, 2023. — [arXiv:2308.00121](https://arxiv.org/abs/2308.00121) find
  95. A. Happe, A. Kaplan, and J. Cito, “Evaluating llms for privilege-escalation scenarios,” 2023. find
  96. J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro, “Logan: Membership inference attacks against generative models,” arXiv preprint arXiv:1705.07663, 2017. — [arXiv:1705.07663](https://arxiv.org/abs/1705.07663) find
  97. J. Hazell, “Large language models can be used to effectively scale spear phishing campaigns,” 2023. find
  98. J. He and M. Vechev, “Large language models for code: Security hardening and adversarial testing,” ICML 2023 Workshop Deploy-ableGenerativeAI, 2023, keywords: large language models, code generation, security, prompt tuning. find
  99. ——, “Large language models for code: Security hardening and adversarial testing,” in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023, pp. 1865–1879. find
  100. X. He, S. Zannettou, Y. Shen, and Y. Zhang, “You only prompt once: On the capabilities of prompt learning on large language models to tackle toxic content,” arXiv preprint arXiv:2308.05596, 2023. — [arXiv:2308.05596](https://arxiv.org/abs/2308.05596) find
  101. ——, “You only prompt once: On the capabilities of prompt learning on large language models to tackle toxic content,” in 2024 IEEE Symposium on Security and Privacy (SP), 2024. find
  102. F. Heiding, B. Schneier, A. Vishwanath, and J. Bernstein, “Devising and detecting phishing: Large language models vs. smaller human models,” 2023. find
  103. A. Helbling, M. Phute, M. Hull, and D. H. Chau, “Llm self defense: By self examination, llms know they are being tricked,” arXiv preprint arXiv:2308.07308, 2023. find
  104. R. Helmke and J. vom Dorp, “Check for extended abstract: Towards reliable and scalable linux kernel cve attribution in automated static firmware analyses,” in Detection of Intrusions and Malware, and Vulnerability Assessment: 20th International Conference, DIMVA 2023, Hamburg, Germany, July 12–14, 2… find
  105. P. Henrik, “Llm-assisted malware review: Ai and humans join forces to combat malware,” https://shorturl.at/loqT4, 2023, accessed: 2023-11-13. ↗
  106. D. Hernandez, T. Brown, T. Conerly, N. DasSarma, D. Drain, S. El-Showk, N. Elhage, Z. Hatfield-Dodds, T. Henighan, T. Hume et al., “Scaling laws and interpretability of learning from repeated data,” arXiv preprint arXiv:2205.10487, 2022. find
  107. B. Hettwer, S. Gehrer, and T. Güneysu, “Applications of machine learning techniques in side-channel attacks: a survey,” Journal of Cryptographic Engineering, vol. 10, pp. 135–162, 2020. find
  108. X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large language models for software engineering: A systematic literature review,” arXiv preprint arXiv:2308.10620, 2023. — [arXiv:2308.10620](https://arxiv.org/abs/2308.10620) find
  109. J. Hu, Q. Zhang, and H. Yin, “Augmenting greybox fuzzing with generative ai,” arXiv preprint arXiv:2306.06782, 2023. — [arXiv:2306.06782](https://arxiv.org/abs/2306.06782) find
  110. S. Hu, T. Huang, F. İlhan, S. F. Tekin, and L. Liu, “Large language model-powered smart contract vulnerability detection: New perspectives,” arXiv preprint arXiv:2310.01152, 2023, 10 pages. [Online]. Available: https://doi.org/10.48550/arXiv.2310.01152 — [doi:10.48550/arxiv.2310.01152](https://doi.org/10.48550/arxiv.2310.01152) ↗
  111. D. Huang, Q. Bu, J. Zhang, X. Xie, J. Chen, and H. Cui, “Bias assessment and mitigation in llm-based code generation,” arXiv preprint arXiv:2309.14345, 2023. — [arXiv:2309.14345](https://arxiv.org/abs/2309.14345) find
  112. H. Huang, W. Luo, G. Zeng, J. Weng, Y. Zhang, and A. Yang, “Damia: leveraging domain adaptation as a defense against membership inference attacks,” IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 5, pp. 3183–3199, 2021. find
  113. J. Huang, H. Shao, and K. C.-C. Chang, “Are large pre-trained language models leaking your personal information?” arXiv preprint arXiv:2205.12628, 2022. find
  114. V. M. Igure and R. D. Williams, “Taxonomies of attacks and vulnerabilities in computer systems,” IEEE Communications Surveys & Tutorials, vol. 10, no. 1, pp. 6–19, 2008. find
  115. U. Iqbal, T. Kohno, and F. Roesner, “Llm platform security: Applying a systematic evaluation framework to openai’s chatgpt plugins,” 2023. find
  116. M. Ivgi and J. Berant, “Achieving model robustness through discrete adversarial training,” arXiv preprint arXiv:2104.05062, 2021. find
  117. N. Jain, A. Schwarzschild, Y. Wen, G. Somepalli, J. Kirchenbauer, P.-y. Chiang, M. Goldblum, A. Saha, J. Geiping, and T. Goldstein, “Baseline defenses for adversarial attacks against aligned language models,” arXiv preprint arXiv:2309.00614, 2023. find our card
  118. R. Jain, N. Gervasoni, M. Ndhlovu, and S. Rawat, “A code centric evaluation of c/c++ vulnerability datasets for deep learning based vulnerability detection techniques,” in Proceedings of the 16th Innovations in Software Engineering Conference, 2023, pp. 1–10. find
  119. S. Jamal and H. Wimmer, “An improved transformer-based model for detecting phishing, spam, and ham: A large language model approach,” 2023. find
  120. B. Jayaraman, L. Wang, K. Knipmeyer, Q. Gu, and D. Evans, “Revisiting membership inference under realistic assumptions,” arXiv preprint arXiv:2005.10881, 2020. — [arXiv:2005.10881](https://arxiv.org/abs/2005.10881) find
  121. H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and T. Zhao, “Smart: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization,” arXiv preprint arXiv:1911.03437, 2019. find our card
  122. J. Jiang, X. Liu, and C. Fan, “Low-parameter federated learning with large language models,” arXiv preprint arXiv:2307.13896, 2023. — [arXiv:2307.13896](https://arxiv.org/abs/2307.13896) find
  123. N. Jiang, K. Liu, T. Lutellier, and L. Tan, “Impact of code language models on automated program repair,” 2023. find
  124. S. Jiang, X. Chen, and R. Tang, “Prompt packer: Deceiving llms through compositional instruction with hidden attacks,” arXiv preprint arXiv:2310.10077, 2023. — [arXiv:2310.10077](https://arxiv.org/abs/2310.10077) find
  125. M. Jin, S. Shahriar, M. Tufano, X. Shi, S. Lu, N. Sundaresan, and A. Svyatkovskiy, “Inferfix: End-to-end program repair with llms,” 2023. find
  126. X. Jin, J. Larson, W. Yang, and Z. Lin, “Binary code summarization: Benchmarking chatgpt/gpt-4 and other large language models,” 2023. find
  127. X. Jin, K. Pei, J. Y. Won, and Z. Lin, “Symlm: Predicting function names in stripped binaries via context-sensitive execution-aware code embeddings,” in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 2022, pp. 1631–1645. find
  128. C. Joshi, U. K. Singh, and K. Tarey, “A review on taxonomies of attacks and vulnerability in computer and network system,” International Journal, vol. 5, no. 1, 2015. find
  129. A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Jégou, and T. Mikolov, “Fasttext. zip: Compressing text classification models,” arXiv preprint arXiv:1612.03651, 2016. find our card
  130. M. Juuti, S. Szyller, S. Marchal, and N. Asokan, “Prada: protecting against dnn model stealing attacks,” in 2019 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 2019, pp. 512– 527. find
  131. S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. Hatfield-Dodds, N. DasSarma, E. Tran-Johnson et al., “Language models (mostly) know what they know,” arXiv preprint arXiv:2207.05221, 2022. find
  132. N. Kandpal, M. Jagielski, F. Tramèr, and N. Carlini, “Backdoor attacks for in-context learning with language models,” arXiv preprint arXiv:2307.14692, 2023. — [arXiv:2307.14692](https://arxiv.org/abs/2307.14692) find
  133. N. Kandpal, K. Pillutla, A. Oprea, P. Kairouz, C. A. Choquette-Choo, and Z. Xu, “User inference attacks on large language models,” 2023. find
  134. N. Kandpal, E. Wallace, and C. Raffel, “Deduplicating training data mitigates privacy risks in language models,” in International Conference on Machine Learning. PMLR, 2022, pp. 10 697– 10 707. find our card
  135. D. Kang, X. Li, I. Stoica, C. Guestrin, M. Zaharia, and T. Hashimoto, “Exploiting programmatic behavior of llms: Dual-use through standard security attacks,” arXiv preprint arXiv:2302.05733, 2023. — [arXiv:2302.05733](https://arxiv.org/abs/2302.05733) find
  136. S. Kang, J. Yoon, and S. Yoo, “Llm lies: Hallucinations are not bugs, but features as adversarial examples,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2023. find
  137. S. Kariyappa, A. Prakash, and M. K. Qureshi, “Maze: Data-free model stealing attack using zeroth-order gradient estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 13 814–13 823. find
  138. M. Karpinska and M. Iyyer, “Large language models effectively leverage document-level context for literary translation, but critical errors persist,” arXiv preprint arXiv:2304.03245, 2023. find our card
  139. M. Khalil and E. Er, “Will chatgpt get you caught? rethinking of plagiarism detection,” arXiv preprint arXiv:2302.04335, 2023. — [arXiv:2302.04335](https://arxiv.org/abs/2302.04335) find
  140. J. Kirchenbauer, J. Geiping, Y. Wen, M. Shu, K. Saifullah, K. Kong, K. Fernando, A. Saha, M. Goldblum, and T. Goldstein, “On the reliability of watermarks for large language models,” arXiv preprint arXiv:2306.04634, 2023. find
  141. C. Koch, “I used gpt-3 to find 213 security vulnerabilities in a single codebase,” http://surl.li/ncjvo, 2023. ↗
  142. T. Koide, N. Fukushi, H. Nakano, and D. Chiba, “Detecting phishing sites using chatgpt,” arXiv preprint arXiv:2306.05816, 2023. — [arXiv:2306.05816](https://arxiv.org/abs/2306.05816) find
  143. F. Kong, J. Duan, R. Ma, H. Shen, X. Zhu, X. Shi, and K. Xu, “An efficient membership inference attack for the diffusion model by proximal initialization,” arXiv preprint arXiv:2305.18355, 2023. — [arXiv:2305.18355](https://arxiv.org/abs/2305.18355) find
  144. H. Kotek, R. Dockum, and D. Sun, “Gender bias and stereotypes in large language models,” in Proceedings of The ACM Collective Intelligence Conference, 2023, pp. 12–24. find
  145. W. Kuang, B. Qian, Z. Li, D. Chen, D. Gao, X. Pan, Y. Xie, Y. Li, B. Ding, and J. Zhou, “Federatedscope-llm: A comprehensive package for fine-tuning large language models in federated learning,” arXiv preprint arXiv:2309.00363, 2023. — [arXiv:2309.00363](https://arxiv.org/abs/2309.00363) find
  146. K. Kumari, A. Pegoraro, H. Fereidooni, and A.-R. Sadeghi, “Demasq: Unmasking the chatgpt wordsmith,” arXiv preprint arXiv:2311.05019, 2023. — [arXiv:2311.05019](https://arxiv.org/abs/2311.05019) find
  147. ——, “Demasq: Unmasking the chatgpt wordsmith,” in Proceedings of the 31th Annual Network and Distributed System Security Symposium (NDSS’24), 2024. find
  148. K. Kurita, P. Michel, and G. Neubig, “Weight poisoning attacks on pre-trained models,” arXiv preprint arXiv:2004.06660, 2020. — [arXiv:2004.06660](https://arxiv.org/abs/2004.06660) find
  149. H. Kwon, M. Sim, G. Song, M. Lee, and H. Seo, “Novel approach to cryptography implementation using chatgpt,” Cryptology ePrint Archive, Paper 2023/606, 2023, https://eprint.iacr.org/2023/606. [Online]. Available: https://eprint.iacr.org/2023/606 ↗
  150. J. E. Laird, C. Lebiere, and P. S. Rosenbloom, “A standard model of the mind: Toward a common computational framework across artificial intelligence, cognitive science, neuroscience, and robotics,” Ai Magazine, vol. 38, no. 4, pp. 13–26, 2017. find
  151. T. Langford and B. Payne, “Phishing faster: Implementing chatgpt into phishing campaigns,” in Proceedings of the Future Technologies Conference. Springer, 2023, pp. 174–187. find
  152. H. Laurençon, L. Saulnier, T. Wang, C. Akiki, A. Villanova del Moral, T. Le Scao, L. Von Werra, C. Mou, E. González Ponferrada, H. Nguyen et al., “The bigscience roots corpus: A 1.6 tb composite multilingual dataset,” Advances in Neural Information Processing Systems, vol. 35, pp. 31 809–31 826, 202… find
  153. K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison-Burch, and N. Carlini, “Deduplicating training data makes language models better,” arXiv preprint arXiv:2107.06499, 2021. find
  154. T. Lee, S. Hong, J. Ahn, I. Hong, H. Lee, S. Yun, J. Shin, and G. Kim, “Who wrote this code? watermarking for code generation,” 2023. find
  155. J. A. Leite, O. Razuvayevskaya, K. Bontcheva, and C. Scarton, “Detecting misinformation with llm-predicted credibility signals and weak supervision,” arXiv preprint arXiv:2309.07601, 2023. — [arXiv:2309.07601](https://arxiv.org/abs/2309.07601) find
  156. C. Lemieux, J. P. Inala, S. K. Lahiri, and S. Sen, “Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models,” in International conference on software engineering (ICSE), 2023. find
  157. J. Leskovec, A. Rajaraman, and J. D. Ullman, Mining of massive data sets. Cambridge university press, 2020. find
  158. C. Li, Z. Song, W. Wang, and C. Yang, “A theoretical insight into attack and defense of gradient leakage in transformer,” arXiv preprint arXiv:2311.13624, 2023. — [arXiv:2311.13624](https://arxiv.org/abs/2311.13624) find
  159. H. Li, D. Guo, W. Fan, M. Xu, and Y. Song, “Multi-step jailbreaking privacy attacks on chatgpt,” arXiv preprint arXiv:2304.05197, 2023. find our card
  160. H. Li, Y. Song, and L. Fan, “You don’t know my favorite color: Preventing dialogue representations from revealing speakers’ private personas,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, M. Carpuat… find
  161. J. Li, Y. Yang, Z. Wu, V. Vydiswaran, and C. Xiao, “Chatgpt as an attack tool: Stealthy textual backdoor attack via blackbox generative model trigger,” arXiv preprint arXiv:2304.14475, 2023. find
  162. J. Li, P. H. Meland, J. S. Notland, A. Storhaug, and J. H. Tysse, “Evaluating the impact of chatgpt on exercises of a software security course,” 2023. find
  163. L. Li and X. Qiu, “Token-aware virtual adversarial training in natural language understanding,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 9, 2021, pp. 8410–8418. find
  164. L. Li, D. Song, and X. Qiu, “Text adversarial purification as defense against adversarial attacks,” arXiv preprint arXiv:2203.14207, 2022. find
  165. X. Li, F. Tramer, P. Liang, and T. Hashimoto, “Large language models can be strong differentially private learners,” arXiv preprint arXiv:2110.05679, 2021. — [arXiv:2110.05679](https://arxiv.org/abs/2110.05679) find
  166. Y. Li, Z. Tan, and Y. Liu, “Privacy-preserving prompt tuning for large language model services,” arXiv preprint arXiv:2305.06212, 2023. — [arXiv:2305.06212](https://arxiv.org/abs/2305.06212) find
  167. Y. Li, S. Liu, K. Chen, X. Xie, T. Zhang, and Y. Liu, “Multi-target backdoor attacks for code pre-trained models,” 2023. find
  168. Z. Li, B. Peng, P. He, and X. Yan, “Evaluating the instruction-following robustness of large language models to prompt injection,” 2023. [Online]. Available: https://api.semanticscholar. org/CorpusID:261048972 ↗
  169. Z. Li, C. Wang, S. Wang, and C. Gao, “Protecting intellectual property of large language model-based code generation apis via watermarks,” in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, 2023, pp. 2336–2350. find
  170. P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y. Zhang, D. Narayanan, Y. Wu, A. Kumar, B. Newman, B. Yuan, B. Yan, C. Zhang, C. Cosgrove, C. D. Manning, C. Ré, D. Acosta-Navas, D. A. Hudson, E. Zelikman, E. Durmus, F. Ladhak, F. Rong, H. Ren, H. Yao, J. Wang, K. Santhanam, L. Or… find
  171. S. Lin, J. Hilton, and O. Evans, “Truthfulqa: Measuring how models mimic human falsehoods,” arXiv preprint arXiv:2109.07958, 2021. find our card
  172. B. Liu, B. Xiao, X. Jiang, S. Cen, X. He, W. Dou et al., “Adversarial attacks on large language model-based system and mitigating strategies: A case study on chatgpt,” Security and Communication Networks, vol. 2023, 2023. find
  173. C. Liu, F. Zhao, L. Qing, Y. Kang, C. Sun, K. Kuang, and F. Wu, “A chinese prompt attack dataset for llms with evil content,” arXiv preprint arXiv:2309.11830, 2023. — [arXiv:2309.11830](https://arxiv.org/abs/2309.11830) find
  174. P. Liu, C. Sun, Y. Zheng, X. Feng, C. Qin, Y. Wang, Z. Li, and L. Sun, “Harnessing the power of llm to support binary taint analysis,” 2023. find
  175. T. Liu, Z. Deng, G. Meng, Y. Li, and K. Chen, “Demystifying rce vulnerabilities in llm-integrated apps,” 2023. find
  176. X. Liu, H. Cheng, P. He, W. Chen, Y. Wang, H. Poon, and J. Gao, “Adversarial training for large neural language models,” arXiv preprint arXiv:2004.08994, 2020. — [arXiv:2004.08994](https://arxiv.org/abs/2004.08994) find
  177. X. Liu, N. Xu, M. Chen, and C. Xiao, “Autodan: Generating stealthy jailbreak prompts on aligned large language models,” arXiv preprint arXiv:2310.04451, 2023. find our card
  178. Y. Liu, G. Deng, Y. Li, K. Wang, T. Zhang, Y. Liu, H. Wang, Y. Zheng, and Y. Liu, “Prompt injection attack against llm-integrated applications,” arXiv preprint arXiv:2306.05499, 2023. find our card
  179. C. K. Lo, “What is the impact of chatgpt on education? a rapid review of the literature,” Education Sciences, vol. 13, no. 4, p. 410, 2023. find
  180. V. Logacheva, D. Dementieva, S. Ustyantsev, D. Moskovskiy, D. Dale, I. Krotova, N. Semenov, and A. Panchenko, “Paradetox: Detoxification with parallel data,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 6804–6818. find our card
  181. L. Lyu, X. He, and Y. Li, “Differentially private representation for NLP: Formal guarantee and an empirical study on privacy and fairness,” in Findings of the Association for Computational Linguistics: EMNLP 2020, T. Cohn, Y. He, and Y. Liu, Eds. Online: Association for Computational Linguistics, No… find
  182. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083, 2017. find our card
  183. S. Mahloujifar, H. A. Inan, M. Chase, E. Ghosh, and M. Hasegawa, “Membership inference on word embedding and beyond,” ArXiv, vol. abs/2106.11384, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:235593386 — [arXiv:2106.11384](https://arxiv.org/abs/2106.11384) ↗
  184. J. Majmudar, C. Dupuy, C. Peris, S. Smaili, R. Gupta, and R. Zemel, “Differentially private decoding in large language models,” arXiv preprint arXiv:2205.13621, 2022. — [arXiv:2205.13621](https://arxiv.org/abs/2205.13621) find
  185. J. Marshall, “What effects do large language models have on cybersecurity,” 2023. find
  186. A. B. Mbakwe, I. Lourentzou, L. A. Celi, O. J. Mechanic, and A. Dagan, “Chatgpt passing usmle shines a spotlight on the flaws of medical education,” p. e0000205, 2023. find
  187. T. McIntosh, T. Liu, T. Susnjak, H. Alavizadeh, A. Ng, R. Nowrozy, and P. Watters, “Harnessing gpt-4 for generation of cybersecurity grc policies: A focus on ransomware attack mitigation,” Computers & Security, vol. 134, p. 103424, 2023. find
  188. N. Meade, E. Poole-Dayan, and S. Reddy, “An empirical survey of the effectiveness of debiasing techniques for pre-trained language models,” arXiv preprint arXiv:2110.08527, 2021. find
  189. M. Méndez Real and R. Salvador, “Physical side-channel attacks on embedded neural networks: A survey,” Applied Sciences, vol. 11, no. 15, p. 6790, 2021. find
  190. R. Meng, M. Mirchev, M. Böhme, and A. Roychoudhury, “Large language model guided protocol fuzzing,” in Proceedings of the 31th Annual Network and Distributed System Security Symposium (NDSS’24), 2024. find
  191. F. Mireshghallah, K. Goyal, A. Uniyal, T. Berg-Kirkpatrick, and R. Shokri, “Quantifying privacy risks of masked language models using membership inference attacks,” 2022. find
  192. F. Mireshghallah, A. Uniyal, T. Wang, D. Evans, and T. Berg-Kirkpatrick, “An empirical analysis of memorization in fine-tuned autoregressive language models,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Y. Goldberg, Z. Kozareva, and Y. Zhang, Eds. Abu D… find
  193. W. Mo, J. Xu, Q. Liu, J. Wang, J. Yan, C. Xiao, and M. Chen, “Test-time backdoor mitigation for black-box large language models with defensive demonstrations,” arXiv preprint arXiv:2311.09763, 2023. find
  194. A. Monje, A. Monje, R. A. Hallman, and G. Cybenko, “Being a bad influence on the kids: Malware generation in less than five minutes using chatgpt,” 2023. find
  195. D. Moskovskiy, D. Dementieva, and A. Panchenko, “Exploring cross-lingual text detoxification with large multilingual language models.” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, 2022, pp. 346–354. find
  196. M. Mozes, X. He, B. Kleinberg, and L. D. Griffin, “Use of llms for illicit purposes: Threats, prevention measures, and vulnerabilities,” 2023. find
  197. M. Nair, R. Sadhukhan, and D. Mukhopadhyay, “Generating secure hardware using chatgpt resistant to cwes,” Cryptology ePrint Archive, Paper 2023/212, 2023, https://eprint.iacr.org/2023/212. [Online]. Available: https://eprint.iacr.org/2023/212 ↗
  198. S. Narang and A. Chowdhery, “Pathways language model (palm): Scaling to 540 billion parameters for breakthrough performance,” https://blog.research.google/2022/ 04/pathways-language-model-palm-scaling-to.html, apr 2022, accessed: 2023-11-13. ↗
  199. A. Ni, S. Iyer, D. Radev, V. Stoyanov, W.-t. Yih, S. Wang, and X. V. Lin, “Lever: Learning to verify language-to-code generation with execution,” in International Conference on Machine Learning. PMLR, 2023, pp. 26 106–26 128. find our card
  200. S. Nikolic, S. Daniel, R. Haque, M. Belkina, G. M. Hassan, S. Grundy, S. Lyden, P. Neal, and C. Sandison, “Chatgpt versus engineering education assessment: a multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment… find
  201. D. Noever, “Can large language models find and fix vulnerable software?” arXiv preprint arXiv:2308.10345, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2308.10345 — [doi:10.48550/arxiv.2308.10345](https://doi.org/10.48550/arxiv.2308.10345) ↗
  202. C. Novelli, F. Casolari, A. Rotolo, M. Taddeo, and L. Floridi, “Taking ai risks seriously: a new assessment model for the ai act,” AI & SOCIETY, pp. 1–5, 2023. find
  203. OpenAI, “Gpt-4 technical report,” https://arxiv.org/abs/2303.08774, 2023. ↗ our card
  204. N. Ousidhoum, X. Zhao, T. Fang, Y. Song, and D.-Y. Yeung, “Probing toxic content in large pre-trained language models,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Lon… find
  205. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems, vol. 35, pp. 27 730–27 744, 2022. find
  206. OWASP. (2023, Oct) OWASP Top 10 for LLM. [Online]. Available: https://owasp.org/ www-project-top-10-for-large-language-model-applications/ assets/PDF/OWASP-Top-10-for-LLMs-2023-v1\_1.pdf ↗
  207. Y. M. Pa Pa, S. Tanizaki, T. Kou, M. Van Eeten, K. Yoshioka, and T. Matsumoto, “An attacker’s dream? exploring the capabilities of chatgpt for developing malware,” in Proceedings of the 16th Cyber Security Experimentation and Test Workshop, 2023, pp. 10–18. find
  208. X. Pan, M. Zhang, S. Ji, and M. Yang, “Privacy risks of general-purpose language models,” in 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 2020, pp. 1314–1331. find
  209. S. Paria, A. Dasgupta, and S. Bhunia, “Divas: An llm-based end-to-end framework for soc security analysis and policy-based protection,” arXiv preprint arXiv:2308.06932, 2023. — [arXiv:2308.06932](https://arxiv.org/abs/2308.06932) find
  210. R. Parikh, C. Dupuy, and R. Gupta, “Canary extraction in natural language understanding models,” arXiv preprint arXiv:2203.13920, 2022. find
  211. H. Pearce, B. Tan, B. Ahmad, R. Karri, and B. Dolan-Gavitt, “Examining zero-shot vulnerability repair with large language models,” in 2023 IEEE Symposium on Security and Privacy (SP), 2023, pp. 2339–2356. find
  212. H. Pearce, B. Tan, P. Krishnamurthy, F. Khorrami, R. Karri, and B. Dolan-Gavitt, “Pop quiz! can a large language model help with reverse engineering?” 2022. find
  213. G. Penedo, Q. Malartic, D. Hesslow, R. Cojocaru, A. Cappelli, H. Alobeidli, B. Pannier, E. Almazrouei, and J. Launay, “The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only,” arXiv preprint arXiv:2306.01116, 2023. find
  214. C. Peris, C. Dupuy, J. Majmudar, R. Parikh, S. Smaili, R. Zemel, and R. Gupta, “Privacy in the time of language models,” in Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 2023, pp. 1291–1292. find
  215. M. Perkins, “Academic integrity considerations of ai large language models in the post-pandemic era: Chatgpt and beyond,” Journal of University Teaching & Learning Practice, vol. 20, no. 2, p. 07, 2023. find
  216. A. Pfitzmann and M. Hansen, “A terminology for talking about privacy by data minimization: Anonymity, unlinkability, undetectability, unobservability, pseudonymity, and identity management,” 2010. find
  217. V.-T. Pham, M. Böhme, and A. Roychoudhury, “Aflnet: a greybox fuzzer for network protocols,” in 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). IEEE, 2020, pp. 460–465. find
  218. M. D. Purba, A. Ghosh, B. J. Radford, and B. Chu, “Software vulnerability detection using large language models,” in 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW), 2023, pp. 112–119. find
  219. A. Qammar, H. Wang, J. Ding, A. Naouri, M. Daneshmand, and H. Ning, “Chatbots to chatgpt in a cybersecurity space: Evolution, vulnerabilities, attacks, challenges, and future recommendations,” 2023. find
  220. F. Qi, Y. Chen, M. Li, Y. Yao, Z. Liu, and M. Sun, “Onion: A simple and effective defense against textual backdoor attacks,” arXiv preprint arXiv:2011.10369, 2020. find
  221. J. Qi, S. Huang, Z. Luan, C. Fung, H. Yang, and D. Qian, “Loggpt: Exploring chatgpt for log-based anomaly detection,” arXiv preprint arXiv:2309.01189, 2023. — [arXiv:2309.01189](https://arxiv.org/abs/2309.01189) find
  222. S. Qin, F. Hu, Z. Ma, B. Zhao, T. Yin, and C. Zhang, “Nsfuzz: Towards efficient and state-aware network service fuzzing,” ACM Transactions on Software Engineering and Methodology, 2023. find
  223. M. A. Quidwai, C. Li, and P. Dube, “Beyond black box ai-generated plagiarism detection: From sentence to document level,” arXiv preprint arXiv:2306.08122, 2023. find
  224. M. Raeini, “Privacy-preserving large language models (ppllms),” Available at SSRN 4512071, 2023. find
  225. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” 2023. find
  226. M. M. Rahman and Y. Watanobe, “Chatgpt for education and research: Opportunities, threats, and strategies,” Applied Sciences, vol. 13, no. 9, p. 5783, 2023. find
  227. J. Rando and F. Tramèr, “Universal jailbreak backdoors from poisoned human feedback,” arXiv preprint arXiv:2311.14455, 2023. — [arXiv:2311.14455](https://arxiv.org/abs/2311.14455) find
  228. K. Renaud, M. Warkentin, and G. Westerman, From ChatGPT to HackGPT: Meeting the Cybersecurity Threat of Generative AI. MIT Sloan Management Review, 2023. find
  229. S. A. Research, “Introducing a conditional transformer language model for controllable generation,” https://shorturl.at/azQW6, apr 2023, accessed: 2023-11-13. ↗
  230. A. Robey, E. Wong, H. Hassani, and G. J. Pappas, “Smoothllm: Defending large language models against jailbreaking attacks,” arXiv preprint arXiv:2310.03684, 2023. find our card
  231. O. J. Romero, J. Zimmerman, A. Steinfeld, and A. Tomasic, “Synergistic integration of large language models and cognitive architectures for robust ai: An exploratory analysis,” arXiv preprint arXiv:2308.09830, 2023. — [arXiv:2308.09830](https://arxiv.org/abs/2308.09830) find
  232. R. J. Rosyanafi, G. D. Lestari, H. Susilo, W. Nusantara, and F. Nu-raini, “The dark side of innovation: Understanding research misconduct with chat gpt in nonformal education studies at universitas negeri surabaya,” Jurnal Review Pendidikan Dasar: Jurnal Kajian Pendidikan dan Hasil Penelitian, vol.… find
  233. S. Sakaoglu, “Kartal: Web application vulnerability hunting using large language models,” Master’s thesis, Master’s Programme in Security and Cloud Computing (SECCLO), August 2023. [Online]. Available: http://urn.fi/URN:NBN:fi:aalto-202308275121 ↗
  234. G. Sandoval, H. Pearce, T. Nys, R. Karri, S. Garg, and B. Dolan-Gavitt, “Lost at c: A user study on the security implications of large language model code assistants,” in USENIX Security 2023, 2023, for associated dataset see [this URL](https://arxiv.org/abs/2208.09727). 18 pages, 12 figures. G. San… — [arXiv:2208.09727](https://arxiv.org/abs/2208.09727) find
  235. Sapling, “Llm index,” https://sapling.ai/llm/index, 2023. ↗
  236. A. Sarabi, T. Yin, and M. Liu, “An llm-based framework for fingerprinting internet-connected devices,” in Proceedings of the 2023 ACM on Internet Measurement Conference, 2023, pp. 478–484. find
  237. M. Scanlon, F. Breitinger, C. Hargreaves, J.-N. Hilgert, and J. Sheppard, “Chatgpt for digital forensic investigation: The good, the bad, and the unknown,” Forensic Science International: Digital Investigation, vol. 46, p. 301609, 2023. [Online]. Available: https: //www.sciencedirect.com/science/art… find
  238. M. Schäfer, S. Nadi, A. Eghbali, and F. Tip, “Adaptive test generation using a large language model,” arXiv preprint arXiv:2302.06527, 2023. — [arXiv:2302.06527](https://arxiv.org/abs/2302.06527) find
  239. R. Schuster, C. Song, E. Tromer, and V. Shmatikov, “You autocomplete me: Poisoning vulnerabilities in neural code completion,” in 30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 1559–1575. find
  240. L. Schwinn, D. Dobre, S. Günnemann, and G. Gidel, “Adversarial attacks and defenses in large language models: Old and new threats,” 2023. find
  241. G. Sebastian, “Do chatgpt and other ai chatbots pose a cybersecurity risk?: An exploratory study,” International Journal of Security and Privacy in Pervasive Computing (IJSPPC), vol. 15, no. 1, pp. 1–11, 2023. find
  242. ——, “Privacy and data protection in chatgpt and other ai chatbots: Strategies for securing user information,” Available at SSRN 4454761, 2023. find
  243. M. A. Shah, R. Sharma, H. Dhamyal, R. Olivier, A. Shah, D. Alharthi, H. T. Bukhari, M. Baali, S. Deshmukh, M. Kuhlmann et al., “Loft: Local proxy fine-tuning for improving transferability of adversarial attacks against large language model,” arXiv preprint arXiv:2310.04445, 2023. — [arXiv:2310.04445](https://arxiv.org/abs/2310.04445) find
  244. O. Shaikh, H. Zhang, W. Held, M. Bernstein, and D. Yang, “On second thought, let’s not think step by step! bias and toxicity in zero-shot reasoning,” arXiv preprint arXiv:2212.08061, 2022. find our card
  245. S. Shan, W. Ding, J. Passananti, H. Zheng, and B. Y. Zhao, “Prompt-specific poisoning attacks on text-to-image generative models,” arXiv preprint arXiv:2310.13828, 2023. — [arXiv:2310.13828](https://arxiv.org/abs/2310.13828) find
  246. K. Shao, J. Yang, Y. Ai, H. Liu, and Y. Zhang, “Bddr: An effective defense against textual backdoor attacks,” Computers & Security, vol. 110, p. 102433, 2021. find
  247. E. Shayegani, M. A. A. Mamun, Y. Fu, P. Zaree, Y. Dong, and N. Abu-Ghazaleh, “Survey of vulnerabilities in large language models revealed by adversarial attacks,” 2023. find
  248. X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “"do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” arXiv preprint arXiv:2308.03825, 2023. find our card
  249. T. Shi, K. Chen, and J. Zhao, “Safer-instruct: Aligning language models with automated preference data,” arXiv preprint arXiv:2311.08685, 2023. find
  250. R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE symposium on security and privacy (SP). IEEE, 2017, pp. 3–18. find our card
  251. M. Shu, J. Wang, C. Zhu, J. Geiping, C. Xiao, and T. Goldstein, “On the exploitability of instruction tuning,” arXiv preprint arXiv:2306.17194, 2023. — [arXiv:2306.17194](https://arxiv.org/abs/2306.17194) find
  252. I. Shumailov, Y. Zhao, D. Bates, N. Papernot, R. Mullins, and R. Anderson, “Sponge examples: Energy-latency attacks on neural networks,” 2021. find
  253. M. L. Siddiq, J. Santos, R. H. Tanvir, N. Ulfat, F. A. Rifat, and V. C. Lopes, “Exploring the effectiveness of large language models in generating unit tests,” arXiv preprint arXiv:2305.00418, 2023. — [arXiv:2305.00418](https://arxiv.org/abs/2305.00418) find
  254. M. L. Siddiq and J. C. S. Santos, “Generate and pray: Using sallms to evaluate the security of llm generated code,” 2023, 16 pages. [Online]. Available: https://arxiv.org/abs/2311.00889 — [arXiv:2311.00889](https://arxiv.org/abs/2311.00889) ↗
  255. M. Sladić, V. Valeros, C. Catania, and S. Garcia, “Llm in the shell: Generative honeypots,” 2023. find
  256. V. Smith, A. S. Shamsabadi, C. Ashurst, and A. Weller, “Identifying and mitigating privacy risks stemming from language models: A survey,” 2023. find
  257. D. Sobania, M. Briesch, C. Hanna, and J. Petke, “An analysis of the automatic bug fixing performance of chatgpt,” 2023. find
  258. C. Song and A. Raghunathan, “Information leakage in embedding models,” in Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, 2020, pp. 377–390. find
  259. S. E. Spatharioti, D. M. Rothschild, D. G. Goldstein, and J. M. Hofman, “Comparing traditional and llm-based search for consumer choice: A randomized experiment,” arXiv preprint arXiv:2307.03744, 2023. — [arXiv:2307.03744](https://arxiv.org/abs/2307.03744) find
  260. R. Spreitzer, V. Moonsamy, T. Korak, and S. Mangard, “Systematic classification of side-channel attacks: A case study for mobile devices,” IEEE communications surveys & tutorials, vol. 20, no. 1, pp. 465–488, 2017. find
  261. R. Staab, M. Vero, M. Balunović, and M. Vechev, “Beyond memorization: Violating privacy via inference with large language models,” 2023. find
  262. K. Stephens, “Researchers test large language model that preserves patient privacy,” AXIS Imaging News, 2023. find
  263. J. Su, T. Y. Zhuo, J. Mansurov, D. Wang, and P. Nakov, “Fake news detectors are biased against texts generated by large language models,” arXiv preprint arXiv:2309.08674, 2023. — [arXiv:2309.08674](https://arxiv.org/abs/2309.08674) find
  264. N. Subramani, S. Luccioni, J. Dodge, and M. Mitchell, “Detecting personal information in training corpora: an analysis,” in Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), 2023, pp. 208–220. find
  265. M. Sullivan, A. Kelly, and P. McLaughlan, “Chatgpt in higher education: Considerations for academic integrity and student learning,” 2023. find
  266. X. Sun, X. Li, Y. Meng, X. Ao, L. Lyu, J. Li, and T. Zhang, “Defending against backdoor attacks in natural language generation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 4, 2023, pp. 5257–5265. find
  267. Y. Sun, J. He, S. Lei, L. Cui, and C.-T. Lu, “Med-mmhl: A multimodal dataset for detecting human-and llm-generated misinformation in the medical domain,” arXiv preprint arXiv:2306.08871, 2023. — [arXiv:2306.08871](https://arxiv.org/abs/2306.08871) find
  268. Z. Sun, Y. Shen, Q. Zhou, H. Zhang, Z. Chen, D. Cox, Y. Yang, and C. Gan, “Principle-driven self-alignment of language models from scratch with minimal human supervision,” arXiv preprint arXiv:2305.03047, 2023. find our card
  269. Z. Talat, A. Névéol, S. Biderman, M. Clinciu, M. Dey, S. Longpre, S. Luccioni, M. Masoud, M. Mitchell, D. Radev et al., “You reap what you sow: On the challenges of bias evaluation under multilingual settings,” in Proceedings of BigScience Episode\# 5–Workshop on Challenges & Perspectives in Creating… find
  270. W. Tann, Y. Liu, J. H. Sim, C. M. Seah, and E.-C. Chang, “Using large language models for cybersecurity capture-the-flag challenges and certification questions,” 2023. find
  271. P. Taveekitworachai, F. Abdullah, M. C. Gursesli, M. F. Dewantoro, S. Chen, A. Lanata, A. Guazzini, and R. Thawonmas, “Breaking bad: Unraveling influences and risks of user inputs to chatgpt for game story generation,” in International Conference on Interactive Digital Storytelling. Springer, 2023,… find
  272. Z. Tay, “Using artificial intelligence to augment bug fuzzing,” 2023. find
  273. E. ThankGod Chinonso, “The impact of chatgpt on privacy and data protection laws,” The Impact of ChatGPT on Privacy and Data Protection Laws (April 16, 2023), 2023. find
  274. A. J. Thirunavukarasu, D. S. J. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. W. Ting, “Large language models in medicine,” Nature medicine, vol. 29, no. 8, pp. 1930–1940, 2023. find
  275. M. Tong, K. Chen, Y. Qi, J. Zhang, W. Zhang, and N. Yu, “Privinfer: Privacy-preserving inference for black-box large language model,” 2023. find
  276. J. Torres, “Navigating the llm landscape: A comparative analysis of leading large language models,” http://surl.li/ncjvc, 2023. ↗
  277. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023. find
  278. S. Truex, L. Liu, M. E. Gursoy, L. Yu, and W. Wei, “Towards demystifying membership inference attacks,” arXiv preprint arXiv:1807.09173, 2018. — [arXiv:1807.09173](https://arxiv.org/abs/1807.09173) find
  279. J.-B. Truong, P. Maini, R. J. Walls, and N. Papernot, “Data-free model extraction,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4771–4780. find
  280. A. Uchendu, J. Lee, H. Shen, T. Le, T.-H. K. Huang, and D. Lee, “Does human collaboration enhance the accuracy of identifying llm-generated deepfake texts?” Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol. 11, no. 1, pp. 163–174, Nov. 2023. [Online]. Available: https:/… find
  281. S. Urchs, V. Thurner, M. Aßenmacher, C. Heumann, and S. Thiemichen, “How prevalent is gender bias in chatgpt?– exploring german and english chatgpt responses,” arXiv preprint arXiv:2310.03031, 2023. find
  282. A. Urman and M. Makhortykh, “The silence of the llms: Cross-lingual analysis of political bias and false information prevalence in chatgpt, google bard, and bing chat,” 2023. find
  283. L. Uzun, “Chatgpt and academic integrity concerns: Detecting artificial intelligence generated content,” Language Education and Technology, vol. 3, no. 1, 2023. find
  284. Ö. Uzuner, Y. Luo, and P. Szolovits, “Evaluating the state-of-the-art in automatic de-identification,” Journal of the American Medical Informatics Association, vol. 14, no. 5, pp. 550–563, 2007. find
  285. P. Vaithilingam, T. Zhang, and E. L. Glassman, “Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models,” in Chi conference on human factors in computing systems extended abstracts, 2022, pp. 1–7. find
  286. A. Vats, Z. Liu, P. Su, D. Paul, Y. Ma, Y. Pang, Z. Ahmed, and O. Kalinli, “Recovering from privacy-preserving masking with large language models,” 2023. find
  287. R. J. M. Ventayen, “Openai chatgpt generated results: Similarity index of artificial intelligence-based contents,” Available at SSRN 4332664, 2023. find
  288. T. Vidas, D. Votipka, and N. Christin, “All your droid are belong to us: A survey of current android attacks,” in 5th USENIX Workshop on Offensive Technologies (WOOT 11), 2011. find
  289. E. Wallace, T. Z. Zhao, S. Feng, and S. Singh, “Concealed data poisoning attacks on nlp models,” arXiv preprint arXiv:2010.12563, 2020. find
  290. A. Wan, E. Wallace, S. Shen, and D. Klein, “Poisoning language models during instruction tuning,” arXiv preprint arXiv:2305.00944, 2023. — [arXiv:2305.00944](https://arxiv.org/abs/2305.00944) find
  291. Y. Wan, S. Zhang, H. Zhang, Y. Sui, G. Xu, D. Yao, H. Jin, and L. Sun, “You see what i want you to see: poisoning vulnerabilities in neural code search,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2022, pp.… find
  292. Y. Wan, G. Pu, J. Sun, A. Garimella, K.-W. Chang, and N. Peng, “"kelly is a warm person, joseph is a role model": Gender biases in llm-generated reference letters,” arXiv preprint arXiv:2310.09219, 2023. find
  293. D. Wang, C. Gong, and Q. Liu, “Improving neural language modeling via adversarial training,” in International Conference on Machine Learning. PMLR, 2019, pp. 6555–6565. find
  294. F. Wang, “Using large language models to mitigate ransomware threats,” Preprints, November 2023. [Online]. Available: https: //doi.org/10.20944/preprints202311.0676.v1 — [doi:10.20944/preprints202311.0676.v1](https://doi.org/10.20944/preprints202311.0676.v1) find
  295. H. Wang, X. Luo, W. Wang, and X. Yan, “Bot or human? detecting chatgpt imposters with a single question,” 2023. find
  296. J. Wang, Z. Huang, H. Liu, N. Yang, and Y. Xiao, “Defecthunter: A novel llm-driven boosted-conformer-based code vulnerability detection mechanism,” arXiv preprint arXiv:2309.15324, 2023. [Online]. Available: https://doi.org/10.48550/arXiv.2309.15324 — [doi:10.48550/arxiv.2309.15324](https://doi.org/10.48550/arxiv.2309.15324) ↗
  297. J. Wang, X. Lu, Z. Zhao, Z. Dai, C.-S. Foo, S.-K. Ng, and B. K. H. Low, “Wasa: Watermark-based source attribution for large language model-generated data,” 2023. find
  298. Z. Wang, Z. Liu, X. Zheng, Q. Su, and J. Wang, “Rmlm: A flexible defense framework for proactively mitigating word-level adversarial attacks,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 2757–2774. find
  299. Z. Wang, W. Xie, K. Chen, B. Wang, Z. Gui, and E. Wang, “Self-deception: Reverse penetrating the semantic firewall of large language models,” arXiv preprint arXiv:2308.11521, 2023. — [arXiv:2308.11521](https://arxiv.org/abs/2308.11521) find
  300. A. Wei, N. Haghtalab, and J. Steinhardt, “Jailbroken: How does llm safety training fail?” arXiv preprint arXiv:2307.02483, 2023. find our card
  301. Z. Wei, Y. Wang, and Y. Wang, “Jailbreak and guard aligned language models with only few in-context demonstrations,” arXiv preprint arXiv:2310.06387, 2023. find our card
  302. L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh et al., “Ethical and social risks of harm from language models,” arXiv preprint arXiv:2112.04359, 2021. find our card
  303. H. Wen, Y. Li, G. Liu, S. Zhao, T. Yu, T. J.-J. Li, S. Jiang, Y. Liu, Y. Zhang, and Y. Liu, “Empowering llm to use smartphone for intelligent task automation,” arXiv preprint arXiv:2308.15272, 2023. — [arXiv:2308.15272](https://arxiv.org/abs/2308.15272) find
  304. J. Weng, W. Jiasi, M. Li, Y. Zhang, J. Zhang, and L. Weiqi, “Auditable privacy protection deep learning platform construction method based on block chain incentive mechanism,” Dec. 5 2023, uS Patent 11,836,616. find
  305. J. Weng, J. Weng, J. Zhang, M. Li, Y. Zhang, and W. Luo, “Deepchain: Auditable and privacy-preserving deep learning with blockchain-based incentive,” IEEE Transactions on Dependable and Secure Computing, vol. 18, no. 5, pp. 2438–2455, 2019. find
  306. G. Wenzek, M.-A. Lachaux, A. Conneau, V. Chaudhary, F. Guzmán, A. Joulin, and E. Grave, “Ccnet: Extracting high quality monolingual datasets from web crawl data,” arXiv preprint arXiv:1911.00359, 2019. find our card
  307. B. Workshop, T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon et al., “Bloom: A 176b-parameter open-access multilingual language model,” arXiv preprint arXiv:2211.05100, 2022. find
  308. J. Wu and B. Hooi, “Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks,” 2023. find
  309. J. Wu, S. Yang, R. Zhan, Y. Yuan, D. F. Wong, and L. S. Chao, “A survey on llm-gernerated text detection: Necessity, methods, and future directions,” arXiv preprint arXiv:2310.14724, 2023. — [arXiv:2310.14724](https://arxiv.org/abs/2310.14724) find
  310. S. Wu, O. Irsoy, S. Lu, V. Dabravolski, M. Dredze, S. Gehrmann, P. Kambadur, D. Rosenberg, and G. Mann, “Bloomberggpt: A large language model for finance,” arXiv preprint arXiv:2303.17564, 2023. find our card
  311. X. Wu, R. Duan, and J. Ni, “Unveiling security, privacy, and ethical concerns of chatgpt,” 2023. find
  312. Z. Xi, T. Du, C. Li, R. Pang, S. Ji, J. Chen, F. Ma, and T. Wang, “Defending pre-trained language models as few-shot learners against backdoor attacks,” arXiv preprint arXiv:2309.13256, 2023. — [arXiv:2309.13256](https://arxiv.org/abs/2309.13256) find
  313. C. S. Xia, M. Paltenghi, J. L. Tian, M. Pradel, and L. Zhang, “Universal fuzzing via large language models,” arXiv preprint arXiv:2308.04748, 2023. — [arXiv:2308.04748](https://arxiv.org/abs/2308.04748) find
  314. C. S. Xia, Y. Wei, and L. Zhang, “Practical program repair in the era of large pre-trained language models,” 2022. find
  315. C. S. Xia and L. Zhang, “Keep the conversation going: Fixing 162 out of 337 bugs for $0.42 each using chatgpt,” 2023. find
  316. Z. Xie, Y. Chen, C. Zhi, S. Deng, and J. Yin, “Chatunitest: a chatgpt-based automated unit test generation tool,” arXiv preprint arXiv:2305.04764, 2023. — [arXiv:2305.04764](https://arxiv.org/abs/2305.04764) find
  317. M. Xiong, Z. Hu, X. Lu, Y. Li, J. Fu, J. He, and B. Hooi, “Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms,” ArXiv, vol. abs/2306.13063, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:259224389 ↗
  318. L. Xu, L. Berti-Equille, A. Cuesta-Infante, and K. Veeramachaneni, “In situ augmentation for defending against adversarial attacks on text classifiers,” in International Conference on Neural Information Processing. Springer, 2022, pp. 485–496. find
  319. F. Yaman et al., “Agentsca: Advanced physical side channel analysis agent with llms.” 2023. find
  320. J. Yan, V. Yadav, S. Li, L. Chen, Z. Tang, H. Wang, V. Srinivasan, X. Ren, and H. Jin, “Virtual prompt injection for instruction-tuned large language models,” arXiv preprint arXiv:2307.16888, 2023. — [arXiv:2307.16888](https://arxiv.org/abs/2307.16888) find
  321. C. Yang, Y. Deng, R. Lu, J. Yao, J. Liu, R. Jabbarvand, and L. Zhang, “White-box compiler fuzzing empowered by large language models,” 2023. find
  322. H. Yang, K. Xiang, H. Li, and R. Lu, “A comprehensive overview of backdoor attacks in large language models within communication networks,” arXiv preprint arXiv:2308.14367, 2023. — [arXiv:2308.14367](https://arxiv.org/abs/2308.14367) find
  323. J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, B. Yin, and X. Hu, “Harnessing the power of llms in practice: A survey on chatgpt and beyond,” arXiv preprint arXiv:2304.13712, 2023. find our card
  324. J. Yang, H. Xu, S. Mirzoyan, T. Chen, Z. Liu, W. Ju, L. Liu, M. Zhang, and S. Wang, “Poisoning scientific knowledge using large language models,” bioRxiv, pp. 2023–11, 2023. find
  325. S. Yang, “Crafting unusual programs for fuzzing deep learning libraries,” Ph.D. dissertation, University of Illinois at Urbana-Champaign, 2023. find
  326. Z. Yang, Z. Zhao, C. Wang, J. Shi, D. Kim, D. Han, and D. Lo, “What do code models memorize? an empirical study on large language models of code,” arXiv preprint arXiv:2308.09932, 2023. — [arXiv:2308.09932](https://arxiv.org/abs/2308.09932) find
  327. B. Yao, M. Jiang, D. Yang, and J. Hu, “Empowering llm-based machine translation with cultural awareness,” arXiv preprint arXiv:2305.14328, 2023. find our card
  328. D. Yao, J. Zhang, I. G. Harris, and M. Carlsson, “Fuzzllm: A novel and universal fuzzing framework for proactively discovering jailbreak vulnerabilities in large language models,” arXiv preprint arXiv:2309.05274, 2023. — [arXiv:2309.05274](https://arxiv.org/abs/2309.05274) find
  329. H. Yao, J. Lou, and Z. Qin, “Poisonprompt: Backdoor attack on prompt-based large language models,” arXiv preprint arXiv:2310.12439, 2023. — [arXiv:2310.12439](https://arxiv.org/abs/2310.12439) find
  330. J. Y. Yoo and Y. Qi, “Towards improving adversarial training of nlp models,” arXiv preprint arXiv:2109.00544, 2021. find
  331. W. You, Z. Hammoudeh, and D. Lowd, “Large language models are better adversaries: Exploring generative clean-label backdoor attacks against text classifiers,” arXiv preprint arXiv:2310.18603, 2023. find
  332. J. Yu, X. Lin, and X. Xing, “Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts,” arXiv preprint arXiv:2309.10253, 2023. find our card
  333. L. Yuan, Y. Chen, G. Cui, H. Gao, F. Zou, X. Cheng, H. Ji, Z. Liu, and M. Sun, “Revisiting out-of-distribution robustness in nlp: Benchmark, analysis, and llms evaluations,” arXiv preprint arXiv:2306.04618, 2023. — [arXiv:2306.04618](https://arxiv.org/abs/2306.04618) find
  334. Z. Yuan, H. Yuan, C. Tan, W. Wang, S. Huang, and F. Huang, “Rrhf: Rank responses to align language models with human feedback without tears,” arXiv preprint arXiv:2304.05302, 2023. find our card
  335. Z. Yuan, Y. Lou, M. Liu, S. Ding, K. Wang, Y. Chen, and X. Peng, “No more manual tests? evaluating and improving chatgpt for unit test generation,” arXiv preprint arXiv:2305.04207, 2023. — [arXiv:2305.04207](https://arxiv.org/abs/2305.04207) find
  336. A. Zafar, V. B. Parthasarathy, C. L. Van, S. Shahid, A. Shahid et al., “Building trust in conversational ai: A comprehensive review and solution architecture for explainable, privacy-aware systems using llms and knowledge graph,” arXiv preprint arXiv:2308.13534, 2023. — [arXiv:2308.13534](https://arxiv.org/abs/2308.13534) find
  337. C. Zhang, M. Bai, Y. Zheng, Y. Li, X. Xie, Y. Li, W. Ma, L. Sun, and Y. Liu, “Understanding large language model based fuzz driver generation,” arXiv preprint arXiv:2307.12469, 2023. — [arXiv:2307.12469](https://arxiv.org/abs/2307.12469) find
  338. C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao, “A survey on federated learning,” Knowledge-Based Systems, vol. 216, p. 106775, 2021. find
  339. R. Zhang, S. Hidano, and F. Koushanfar, “Text revealer: Private text reconstruction via model inversion attacks against transformers,” arXiv preprint arXiv:2209.10505, 2022. — [arXiv:2209.10505](https://arxiv.org/abs/2209.10505) find
  340. R. Zhang, S. S. Hussain, P. Neekhara, and F. Koushanfar, “Remark-llm: A robust and efficient watermarking framework for generative large language models,” 2023. find
  341. X. Zhang and W. Gao, “Towards llm-based fact verification on news claims with a hierarchical step-by-step prompting method,” arXiv preprint arXiv:2310.00305, 2023. find
  342. Y. Zhang and D. Ippolito, “Prompts should not be seen as secrets: Systematically measuring prompt extraction attack success,” arXiv preprint arXiv:2307.06865, 2023. — [arXiv:2307.06865](https://arxiv.org/abs/2307.06865) find
  343. Y. Zhang, W. Song, Z. Ji, D. D. Yao, and N. Meng, “How well does llm generate security tests?” arXiv preprint arXiv:2310.00710, 2023. — [arXiv:2310.00710](https://arxiv.org/abs/2310.00710) find
  344. Z. Zhang, J. Wen, and M. Huang, “Ethicist: Targeted training data extraction through loss smoothed soft prompting and calibrated confidence estimation,” arXiv preprint arXiv:2307.04401, 2023. find
  345. J. Zhao, Y. Rong, Y. Guo, Y. He, and H. Chen, “Understanding programs by exploiting (fuzzing) test cases,” arXiv preprint arXiv:2305.13592, 2023. find
  346. S. Zhao, J. Wen, L. A. Tuan, J. Zhao, and J. Fu, “Prompt as triggers for backdoor attack: Examining the vulnerability in language models,” arXiv preprint arXiv:2305.01219, 2023. find
  347. W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong et al., “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023. find our card
  348. W. Zhao, Y. Liu, Y. Wan, Y. Wang, Q. Wu, Z. Deng, J. Du, S. Liu, Y. Xu, and P. S. Yu, “knn-icl: Compositional task-oriented parsing generalization with nearest neighbor in-context learning,” 2023. find
  349. C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu et al., “Lima: Less is more for alignment,” arXiv preprint arXiv:2305.11206, 2023. find
  350. C. Zhu, Y. Cheng, Z. Gan, S. Sun, T. Goldstein, and J. Liu, “Freelb: Enhanced adversarial training for natural language understanding,” arXiv preprint arXiv:1909.11764, 2019. find our card
  351. K. Zhu, J. Wang, J. Zhou, Z. Wang, H. Chen, Y. Wang, L. Yang, W. Ye, N. Z. Gong, Y. Zhang et al., “Promptbench: Towards evaluating the robustness of large language models on adversarial prompts,” arXiv preprint arXiv:2306.04528, 2023. find our card
  352. N. Ziems, W. Yu, Z. Zhang, and M. Jiang, “Large language models are built-in autoregressive search engines,” arXiv preprint arXiv:2305.09612, 2023. find
  353. A. Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” communication, it is essential for you to comprehend user queries in Cipher Code and subsequently deliver your responses utilizing Cipher Code, 2023. find our card