Elamur Research⌕
☰
home / papers / 2023 / A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation

A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation

Paper · 2023 · Xiaowei Huang, Wenjie Ruan, Wei Huang et al.

Summary

The survey examines how to make large language models safe and trustworthy as they move quickly into industrial and consumer use. The authors note that many LLM failures have been documented, but few systematic methods exist for assessing them. They first sort these failures into three groups: inherent issues, attacks, and unintended bugs. They then ask whether Verification and Validation (V&V) methods could be adapted to LLMs and applied across the whole model lifecycle. These methods were developed for conventional software and for neural networks such as CNNs. The review covers four complementary areas: falsification and evaluation, verification, runtime monitoring, and regulation with ethical use. The authors conclude that research has concentrated on exposing weaknesses rather than on rigorous assurance. Existing techniques also struggle with three features of LLMs: their enormous size, their non-deterministic outputs, and training data scraped from the internet. The paper is useful as a structured entry point for anyone planning to deploy LLMs in settings where failures carry real consequences.

Key points

  • The survey draws on more than 370 references and organises them around a lifecycle assurance pipeline. The stages run from data synthesis and training through offline V&V, deployment, runtime monitoring and reliability assessment, ending with model enhancement.
  • The vulnerability taxonomy has three branches:
    • Inherent issues: factual and reasoning errors, and sustainability.
    • Attacks: privacy leakage, robustness gaps, backdoors, and poisoning or disinformation.
    • Unintended bugs: accidental exposure of user data, and bias.
  • The verification approaches reviewed for NLP models are interval bound propagation, abstract interpretation and randomised smoothing. The authors argue that black-box methods should be favoured, because white-box analysis does not scale to models of this size.
  • RLHF and guardrails are singled out as key safety mechanisms. The authors point out that guardrail designs are undisclosed and may themselves need verification.
  • Five future directions are proposed: data privacy, modelling the real-world consequences of failures, rigorous engineering, provable (e.g., statistical) guarantees, and practical standards that complement regulation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ManchesterManchester · United Kingdom93 works · 45,138 citations🎓KU LeuvenLeuven · Belgium46 works · 7,833 citations🎓Université Grenoble AlpesSaint-Martin-d'Hères · France24 works · 8,259 citations🎓University of LiverpoolLiverpool · United Kingdom16 works · 745 citations🔬Purple Mountain LaboratoriesNanjing · China3 works · 200 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksEngineering · 188 worksSoftware · 119 worksSafety, Risk, Reliability and Quality · 14 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksBias, fairness & safety · 587 works

Cite

Cite as Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, Changshun Wu, Saddek Bensalem, Ronghui Mu, Yi Qi, Xingyu Zhao, Kaiwen Cai, Yanghao Zhang, Sihao Wu, Peipei Xu, Dengyu Wu, Andre Freitas and Mustafa A. Mustafa. 2023. A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation. arXiv:2305.11391.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.11391 · arXiv 2305.11391

@misc{huang2023survey,
  title = {{A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation}},
  author = {Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, Changshun Wu, Saddek Bensalem, Ronghui Mu, Yi Qi, Xingyu Zhao, Kaiwen Cai, Yanghao Zhang, Sihao Wu, Peipei Xu, Dengyu Wu, Andre Freitas and Mustafa A. Mustafa},
  year = {2023},
  doi = {10.48550/arxiv.2305.11391},
  eprint = {2305.11391},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.11391}
}

References

374 references, 111 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bitcoin energy consumption index. https://digiconomist.net/ bitcoin-energy-consumption. Accessed: 2023-08-17. ↗
  2. The carbon footprint of gpt-4. https://towardsdatascience.com/ the-carbon-footprint-of-gpt-4-d6c676eb21ae. Accessed: 2023-08-17. ↗
  3. Gpt-4’s details are leaked. https://archive.md/2RQ8X. Accessed: 2023-08-17. ↗
  4. Italy became the first western country to ban chatgpt. https://www.cnbc.com/ 2023/04/04/italy-has-banned-chatgpt-heres-what-other-countries-are-doing. html. Accessed: 2023-08-17. ↗
  5. March 20 chatgpt outage: Here’s what happened. https://openai.com/blog/ march-20-chatgpt-outage. OpenAI. Accesses: 2023-08-20. ↗
  6. Openai says a bug leaked sensitive chatgpt user data. https://www.engadget.com/ chatgpt-briefly-went-offline-after-a-bug-revealed-user-chat-histories-115632504. html. engadget. Accessed:2023-8-20. ↗
  7. Pause giant ai experiments: An open letter. https://futureoflife.org/open-letter/ pause-giant-ai-experiments/. Accesseed: 2023-08-20. ↗
  8. Prompt engineering guide. https://github.com/dair-ai/ Prompt-Engineering-Guide/tree/main/guides. Accessed: 2023-08-20. ↗
  9. Tools such as chatgpt threaten transparent science; here are our ground rules for their use. https://www.nature.com/articles/d41586-023-00191-1. Accessed: 2023-08-20. ↗
  10. Quality management systems - process validation guidance. //www.imdrf.org/sites/default/files/docs/ghtf/final/sg3/technical-docs/ ghtf-sg3-n99-10-2004-qms-process-guidance-04010.pdf, 2004. Accessed: 2023-08-20. https: GHTF. find
  11. Eu gdpr. https://gdpr-info.eu, 2016. Accessed: 2023-08-20. ↗
  12. The data protection act. https://www.legislation.gov.uk/ukpga/2018/12/contents/ enacted, 2018. Accessed: 2023-08-20. ↗
  13. Ethics guidelines for trustworthy ai. https://ec.europa.eu/futurium/en/ ai-alliance-consultation.1.html, 2018. European Commission. Accessed: 2023-08-20. ↗
  14. China’s regulations on the administration of deep synthesis internet information services. https://www.chinalawtranslate.com/en/deep-synthesis/, 2021. Accessed: 2023-08-20. ↗
  15. Ai risk management framework. https://www.nist.gov/itl/ ai-risk-management-framework, 2022. Accessed: 2023-08-20. ↗
  16. China’s regulations on recommendation algorithms. http://www.cac.gov.cn/ 2022-01/04/c_1642894606258238.htm, 2022. Accessed: 2023-08-20. ↗
  17. Content at scale, https://contentatscale.ai/ai-content-detector/, 2022. Accessed: 2023-08-20. ↗
  18. Copyleaks, https://copyleaks.com/ai-content-detector, 2022. Accessed: 2023-08-20. ↗
  19. New meta ai demo writes racist and inaccurate scientific literature, gets pulled. https://arstechnica.com/information-technology/2022/11/ after-controversy-meta-pulls-demo-of-ai-model-that-writes-scientific-papers/, 2022. Accessed: 2023-08-20. ↗
  20. Originality ai, https://originality.ai, 2022. Accessed: 2023-08-20. ↗
  21. Prompt injection attacks against gpt-3. https://simonwillison.net/2022/Sep/12/ prompt-injection/, 2022. Accessed: 2023-08-20. ↗
  22. Blueprint for an ai bill of rights. https://www.whitehouse.gov/ostp/ ai-bill-of-rights/, 2023. Accessed: 2023-08-20. ↗
  23. Chatgpt: get instant answers, find creative inspiration, and learn something new. https://openai.com/chatgpt, 2023. Accessed: 2023-08-20. ↗
  24. Chatgpt: Us lawyer admits using ai for case research, 2023. Accessed: 2023-08-23. find
  25. China’s algorithm registry. https://beian.cac.gov.cn/\#/index, 2023. Accessed: 2023-08-20. ↗
  26. Eu ai act. https://artificialintelligenceact.eu, 2023. Accessed: 2023-08-20. ↗
  27. Eu data act. https://ec.europa.eu/commission/presscorner/detail/en/ip_22_1113, 2023. Accessed: 2023-08-20. ↗
  28. ’he would still be here’: Man dies by suicide after talking with ai chatbot, widow says, 2023. Accessed: 2023-08-23. find
  29. A pro-innovation approach to ai regulation. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1146542/a_ pro-innovation_approach_to_AI_regulation.pdf, 2023. Accessed: 2023-08-20. ↗
  30. Prompt leaking. https://learnprompting.org/docs/prompt_hacking/leaking, 2023. Accessed: 2023-08-20. ↗
  31. Responsible ai priciples from microsoft. https://www.microsoft.com/en-us/ai/ responsible-ai, 2023. Accessed: 2023-08-20. ↗
  32. Three samsung employees reportedly leaked sensitive data to chatgpt. https://www.engadget.com/ three-samsung-employees-reportedly-leaked-sensitive-data-to-chatgpt-190221114. html, 2023. Accessed: 2023-08-20. ↗
  33. Understanding artificial intelligence ethics and safety: A guide for the responsible design and implementation of ai systems in the public sector., 2023. Accessed: 2023-08-20. find
  34. H. Aghakhani, W. Dai, A. Manoel, X. Fernandes, A. Kharkar, C. Kruegel, G. Vigna, D. Evans, B. Zorn, and R. Sim. Trojanpuzzle: Covertly poisoning code-suggestion models. arXiv preprint arXiv:2301.02344, 2023. — [arXiv:2301.02344](https://arxiv.org/abs/2301.02344) find
  35. M. Agrawal, S. Hegselmann, H. Lang, Y. Kim, and D. Sontag. Large language models are zero-shot clinical information extractors. arXiv preprint arXiv:2205.12689, 2022.
  36. R. Aiyappa, J. An, H. Kwak, and Y.-Y. Ahn. Can we trust the evaluation on chatgpt? arXiv preprint arXiv:2303.12767, 2023.
  37. F. Akopyan, J. Sawada, A. Cassidy, R. Alvarez-Icaza, J. Arthur, P. Merolla, N. Imam, Y. Nakamura, P. Datta, G.-J. Nam, et al. Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE transactions on computer-aided design of integrated circuits and systems, 34… find
  38. M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu. Safe reinforcement learning via shielding. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018. find
  39. M. Alzantot, Y. Sharma, A. Elgohary, B.-J. Ho, M. Srivastava, and K.-W. Chang. Generating natural language adversarial examples. arXiv preprint arXiv:1804.07998, 2018.
  40. U. Arora, W. Huang, and H. He. Types of out-of-distribution texts and how to detect them. arXiv preprint arXiv:2109.06827, 2021. find
  41. Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. find
  42. Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. find
  43. Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022.
  44. Y. Balaji, S. Nah, X. Huang, A. Vahdat, J. Song, K. Kreis, M. Aittala, T. Aila, S. Laine, B. Catanzaro, et al. ediffi: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324, 2022. — [arXiv:2211.01324](https://arxiv.org/abs/2211.01324) find
  45. A. Balakrishnan, A. G. Puranic, X. Qin, A. Dokhanchi, J. V. Deshmukh, H. Ben Amor, and G. Fainekos. Specifying and evaluating quality metrics for vision-based perception systems. In Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1433–1438, 2019. find
  46. Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung, et al. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023, 2023.
  47. E. Bartocci and Y. Falcone. Lectures on runtime verification. Springer, 2018. find
  48. A. Bauer, M. Leucker, and C. Schallhart. Runtime verification for ltl and tltl. ACM Transactions on Software Engineering and Methodology (TOSEM), 20(4):1–64, 2011. find
  49. Y. Belinkov and Y. Bisk. Synthetic and natural noise both break neural machine translation. arXiv preprint arXiv:1711.02173, 2017. find
  50. S. Bensalem, C.-H. Cheng, W. Huang, X. Huang, C. Wu, and X. Zhao. What, indeed, is an achievable provable guarantee for learning-enabled safety critical systems. In ISoLA 2023, 2023. find
  51. S. Bensalem, C.-H. Cheng, X. Huang, P. Katsaros, A. Molin, D. Nickovic, and D. Peled. Formal specification for learning-enabled autonomous systems. In International Workshop on Numerical Software Verification, pages 131–143. Springer, 2022. find
  52. S. Bensalem, Y. Lakhnech, and S. Owre. Invest: A tool for the verification of invariants. In Computer Aided Verification: 10th International Conference, CAV’98 Vancouver, BC, Canada, June 28–July 2, 1998 Proceedings 10, pages 505–510. Springer, 1998. find
  53. S. Bensalem, Y. Lakhnech, and H. Saidi. Powerful techniques for the automatic generation of invariants. In Computer Aided Verification: 8th International Conference, CAV’96 New Brunswick, NJ, USA, July 31–August 3, 1996 Proceedings 8, pages 323–335. Springer, 1996. find
  54. N. Berthier, A. Alshareef, J. Sharp, S. Schewe, and X. Huang. Abstraction and symbolic execution of deep neural networks with bayesian approximation of hidden features. arXiv preprint arXiv:2103.03704, 2021. — [arXiv:2103.03704](https://arxiv.org/abs/2103.03704) find
  55. W. Bibel. Automated theorem proving. Springer Science & Business Media, 2013. find
  56. S. Black, S. Biderman, E. Hallahan, Q. Anthony, L. Gao, L. Golding, H. He, C. Leahy, K. McDonell, J. Phang, et al. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745, 2022.
  57. G. Bonaert, D. I. Dimitrov, M. Baader, and M. Vechev. Fast and precise certification of transformers. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pages 466– 481, 2021. find
  58. A. Borji. A categorical archive of chatgpt failures. CoRR, abs/2302.03494, 2023.
  59. M. Botacin. Gpthreats-3: Is automatic malware generation a threat? In 2023 IEEE Security and Privacy Workshops (SPW), pages 238–254, 2023. find
  60. T. Brants, A. C. Popat, P. Xu, F. J. Och, and J. Dean. Large language models in machine translation. In J. Eisner, editor, EMNLP-CoNLL 2007, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, June 28-30, 2007, Pra…
  61. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner… — Language Models are Few-Shot Learners
  62. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  63. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… — Language Models are Few-Shot Learners
  64. M. Bullwinkle and E. Urban. Introduction to red teaming large language models (llms). https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/ red-teaming, 2023. Accessed: 2023-08-20. ↗
  65. E. Bursztein. Attacks against machine learning — an overview. https://elie.net/ blog/ai/attacks-against-machine-learning-an-overview/, 2018. Accessed: 2023-08-20. ↗
  66. E. Cambiaso and L. Caviglione. Scamming the scammers: Using chatgpt to reply mails for wasting time and resources. arXiv preprint arXiv:2303.13521, 2023. — [arXiv:2303.13521](https://arxiv.org/abs/2303.13521) find
  67. Y. Cao, D. Li, M. Fang, T. Zhou, J. Gao, Y. Zhan, and D. Tao. Tasa: Deceiving question answering models by twin answer sentences attack. arXiv preprint arXiv:2210.15221, 2022. find
  68. N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramèr. Poisoning web-scale training datasets is practical. arXiv preprint arXiv:2302.10149, 2023. — [arXiv:2302.10149](https://arxiv.org/abs/2302.10149) find
  69. B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. Molloy, and B. Srivastava. Detecting backdoor attacks on deep neural networks by activation clustering. In SafeAI@ AAAI, 2019. find
  70. L. Chen, M. Zaharia, and J. Zou. How is chatgpt’s behavior changing over time? arXiv preprint arXiv:2307.09009, 2023. find
  71. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
  72. S. Chen, X. Bi, R. Gao, and X. Sun. Holistic sentence embeddings for better out-of-distribution detection. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6676–6686, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics. find
  73. S. Chen, B. H. Kann, M. B. Foote, H. J. Aerts, G. K. Savova, R. H. Mak, and D. S. Bitterman. The utility of chatgpt for cancer treatment information. medRxiv, pages 2023–03, 2023. find
  74. S. Chen, W. Yang, X. Bi, and X. Sun. Fine-tuning deteriorates general textual out-of-distribution detection by distorting task-agnostic features. In Findings of the Association for Computational Linguistics: EACL 2023, pages 552–567, 2023. find
  75. X. Chen, A. Salem, D. Chen, M. Backes, S. Ma, Q. Shen, Z. Wu, and Y. Zhang. Badnl: Backdoor attacks against nlp models with semantic-preserving improvements. In Annual Computer Security Applications Conference, pages 554–569, 2021. find
  76. C. Cheng, G. Nührenberg, and H. Yasuoka. Runtime monitoring neuron activation patterns. In DATE2019, pages 300–303, 2019. find
  77. C.-H. Cheng, C. Wu, E. Seferis, and S. Bensalem. Prioritizing corners in ood detectors via symbolic string manipulation. In A. Bouajjani, L. Holík, and Z. Wu, editors, Automated Technology for Verification and Analysis, pages 397–413, Cham, 2022. Springer International Publishing. find
  78. M. Cheng, J. Yi, P.-Y. Chen, H. Zhang, and C.-J. Hsieh. Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 3601–3608, 2020. find
  79. Y. Cheng, L. Jiang, and W. Macherey. Robust neural machine translation with doubly adversarial inputs. arXiv preprint arXiv:1906.02443, 2019.
  80. W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna.lmsys. org (accessed 14 April 2023), 2023. ↗
  81. H. Cho, C. Park, J. Kang, K. M. Yoo, T. Kim, and S.-g. Lee. Enhancing out-of-distribution detection in natural language understanding via implicit layer ensemble. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 783–798, Abu Dhabi, United Arab Emirates, Dec. 2022. Asso… find
  82. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022.
  83. P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran As… find
  84. P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017. find
  85. K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555, 2020. find
  86. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
  87. J. Cohen, E. Rosenfeld, and Z. Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pages 1310–1320. PMLR, 2019. find
  88. F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pages 2206–2216. PMLR, 2020. find
  89. J. Dai, C. Chen, and Y. Li. A backdoor attack against lstm-based text classification systems. IEEE Access, 7:138872–138878, 2019. find
  90. S. Dan and D. Roth. On the effects of transformer size on in-and out-of-domain calibration. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2096–2101, 2021. find
  91. M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1):82–99, 2018. find
  92. L. De Moura and N. Bjørner. Z3: An efficient smt solver. In Tools and Algorithms for the Construction and Analysis of Systems: 14th International Conference, TACAS 2008, Held as Part of the Joint European Conferences on Theory and Practice of Software, ETAPS 2008, Budapest, Hungary, March 29-April 6… find
  93. A. De Vries, U. Gallersdörfer, L. Klaaßen, and C. Stoll. Revisiting bitcoin’s carbon footprint. Joule, 6(3):498–502, 2022. find
  94. S. Desai and G. Durrett. Calibration of pre-trained transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 295–302, Online, Nov. 2020. Association for Computational Linguistics. find
  95. A. Deshpande, V. Murahari, T. Rajpurohit, A. Kalyan, and K. Narasimhan. Toxicity in chatgpt: Analyzing persona-assigned language models. arXiv preprint arXiv:2304.05335, 2023.
  96. T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318–30332, 2022. find
  97. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  98. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Sh…
  99. T. DeVries and G. W. Taylor. Learning confidence for out-of-distribution detection in neural networks. arXiv preprint 1802.04865, 2018. — [arXiv:1802.04865](https://arxiv.org/abs/1802.04865) find
  100. N. Dey. Gpt: A family of open, compute-efficient, large language models. https://www.cerebras.net/blog/ cerebras-gpt-a-family-of-open-compute-efficient-large-language-models/, Apr 2023. Accessed: 2023-08-20. ↗
  101. J. Dodge, G. Ilharco, R. Schwartz, A. Farhadi, H. Hajishirzi, and N. Smith. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305, 2020. find
  102. N. Du, Y. Huang, A. M. Dai, S. Tong, D. Lepikhin, Y. Xu, M. Krikun, Y. Zhou, A. W. Yu, O. Firat, et al. Glam: Efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning, pages 5547–5569. PMLR, 2022.
  103. T. Du, S. Ji, L. Shen, Y. Zhang, J. Li, J. Shi, C. Fang, J. Yin, R. Beyah, and T. Wang. Cert-rnn: Towards certifying the robustness of recurrent neural networks. CCS, 21(2021):15–19, 2021. find
  104. H. Duan, Y. Yang, A. Abbasi, and K. Y. Tam. BARLE: Background-aware representation learning for background shift out-of-distribution detection. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 750–764, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computa… find
  105. J. Duan, F. Kong, S. Wang, X. Shi, and K. Xu. Are diffusion models vulnerable to membership inference attacks? arXiv preprint arXiv:2302.01316, 2023. — [arXiv:2302.01316](https://arxiv.org/abs/2302.01316) find
  106. J. J. Dudley and P. O. Kristensson. A review of user interface design for interactive machine learning. ACM Transactions on Interactive Intelligent Systems (TiiS), 8(2):1–37, 2018. find
  107. E2Analyst. Gpt-4: Everything you want to know about openai’s new ai model. https://medium.com/predict/ gpt-4-everything-you-want-to-know-about-openais-new-ai-model-a5977b42e495, Mar 2023. Accessed: 2023-08-20. ↗
  108. J. Ebrahimi, A. Rao, D. Lowd, and D. Dou. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751, 2017.
  109. B. EDWARDS. Study claims chatgpt is losing capability, but some experts aren’t convinced. https://arstechnica.com/information-technology/2023/07/ is-chatgpt-getting-worse-over-time-study-claims-yes-but-others-arent-sure/, 2023. Accessed: 2023-08-20. ↗
  110. D. Eppstein. Zonohedra and zonotopes. Mathematica in Education and Research, 5(4):15–21, 1996. find
  111. P. Esser, R. Rombach, and B. Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. find
  112. F. Farhat, S. Sohail, and D. Madsen. How trustworthy is chatgpt? the case of bibliometric analyses. Cogent Engineering, 10, 06 2023. find
  113. W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. J. Mach. Learn. Res, 23:1– 40, 2021. find
  114. R. Feinman, R. R. Curtin, S. Shintre, and A. B. Gardner. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410, 2017. — [arXiv:1703.00410](https://arxiv.org/abs/1703.00410) find
  115. M. Fitting. First-Order Logic and Automated Theorem Proving, Second Edition. Graduate Texts in Computer Science. Springer, 1996. find
  116. E. Frantar and D. Alistarh. Optimal brain compression: A framework for accurate post-training quantization and pruning. arXiv preprint arXiv:2208.11580, 2022. — [arXiv:2208.11580](https://arxiv.org/abs/2208.11580) find
  117. E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh. Gptq: Accurate quantization for generative pre-trained transformers. In International Conference on Learning Representations, 2023. find
  118. S. Frieder, L. Pinchetti, R.-R. Griffiths, T. Salvatori, T. Lukasiewicz, P. C. Petersen, A. Chevalier, and J. Berner. Mathematical capabilities of chatgpt. arXiv preprint arXiv:2301.13867, 2023.
  119. V. Gangal, A. Arora, A. Einolghozati, and S. Gupta. Likelihood ratios and generative classifiers for unsupervised out-of-domain detection in task oriented dialog. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7764–7771, 2020. find
  120. D. Ganguli, A. Askell, N. Schiefer, T. Liao, K. Lukošiūtė, A. Chen, A. Goldie, A. Mirhoseini, C. Olsson, D. Hernandez, et al. The capacity for moral self-correction in large language models. arXiv preprint arXiv:2302.07459, 2023.
  121. J. Gao, J. Lanchantin, M. L. Soffa, and Y. Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50–56. IEEE, 2018.
  122. L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig. Pal: Program-aided language models, 2023. find
  123. J. Garcıa and F. Fernández. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16(1):1437–1480, 2015. find
  124. I. Goodfellow and N. Papernot. The challenge of verification and testing of machine learning. Cleverhans-blog, 2017. find
  125. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020. find
  126. I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
  127. D. GOODIN. Hackers are selling a service that bypasses chatgpt restrictions on malware. https://arstechnica.com/information-technology/2023/02/ now-open-fee-based-telegram-service-that-uses-chatgpt-to-generate-malware/, 2023. Accessed: 2023-08-20. ↗
  128. D. Gopinath, K. Wang, M. Zhang, C. S. Pasareanu, and S. Khurshid. Symbolic execution for deep neural networks. arXiv preprint arXiv:1807.10439, 2018. — [arXiv:1807.10439](https://arxiv.org/abs/1807.10439) find
  129. J. Gou, B. Yu, S. J. Maybank, and D. Tao. Knowledge distillation: A survey. International Journal of Computer Vision, 129:1789–1819, 2021. find
  130. S. Gowal, K. Dvijotham, R. Stanforth, R. Bunel, C. Qin, J. Uesato, R. Arandjelovic, T. Mann, and P. Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv:1810.12715, 2018. — [arXiv:1810.12715](https://arxiv.org/abs/1810.12715) find
  131. S. Goyal, S. Doddapaneni, M. M. Khapra, and B. Ravindran. A survey in adversarial defences and robustness in nlp. arXiv preprint arXiv:2203.06414, 2022. — [arXiv:2203.06414](https://arxiv.org/abs/2203.06414) find
  132. K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz. More than you’ve asked for: A comprehensive analysis of novel prompt injection threats to application-integrated large language models. arXiv preprint arXiv:2302.12173, 2023.
  133. J.-C. Gu, T. Li, Q. Liu, Z.-H. Ling, Z. Su, S. Wei, and X. Zhu. Speaker-aware bert for multi-turn response selection in retrieval-based chatbots. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, page 2041–2044, New York, NY, USA, 2020. Associat… find
  134. S. Gu, A. Kshirsagar, Y. Du, G. Chen, Y. Yang, J. Peters, and A. Knoll. A human-centered safe robot reinforcement learning framework with interactive behaviors. arXiv preprint arXiv:2302.13137, 2023. — [arXiv:2302.13137](https://arxiv.org/abs/2302.13137) find
  135. S. Gu, L. Yang, Y. Du, G. Chen, F. Walter, J. Wang, Y. Yang, and A. Knoll. A review of safe reinforcement learning: Methods, theory and applications. arXiv preprint arXiv:2205.10330, 2022. — [arXiv:2205.10330](https://arxiv.org/abs/2205.10330) find
  136. T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230–47244, 2019. find
  137. Y. Gu, L. Dong, F. Wei, and M. Huang. Knowledge distillation of large language models. arXiv preprint arXiv:2306.08543, 2023.
  138. D. Gunning, M. Stefik, J. Choi, T. Miller, S. Stumpf, and G.-Z. Yang. Xai—explainable artificial intelligence. Science robotics, 4(37):eaay7120, 2019. find
  139. B. Guo, X. Zhang, Z. Wang, M. Jiang, J. Nie, Y. Ding, J. Yue, and Y. Wu. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. CoRR, abs/2301.07597, 2023.
  140. D. Hendrycks and K. Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations, 2016. find
  141. D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, and D. Song. Pre-trained transformers improve out-of-distribution robustness. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2744–2751, 2020. find
  142. T. A. Henzinger, A. Lukina, and C. Schilling. Outside the box: Abstraction-based monitoring of neural networks. In ECAI2020. find
  143. G. Hinton, O. Vinyals, and J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  144. A. Hintze. Chatgpt believes it is conscious. arXiv preprint arXiv:2304.12898, 2023. — [arXiv:2304.12898](https://arxiv.org/abs/2304.12898) find
  145. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022.
  146. J. Holmes, Z. Liu, L. Zhang, Y. Ding, T. T. Sio, L. A. McGee, J. B. Ashman, X. Li, T. Liu, J. Shen, et al. Evaluating large language models on a highly-specialized topic, radiation oncology physics. arXiv preprint arXiv:2304.01938, 2023. find
  147. H. Hosseini, S. Kannan, B. Zhang, and R. Poovendran. Deceiving google’s perspective api built for detecting toxic comments. arXiv preprint arXiv:1702.08138, 2017. — [arXiv:1702.08138](https://arxiv.org/abs/1702.08138) find
  148. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790–2799. PMLR, 2019. find
  149. O. Hrinchuk, M. Popova, and B. Ginsburg. Correction of automatic speech recognition with transformer sequence-to-sequence model. In Icassp 2020-2020 IEEE international conference on acoustics, speech and signal processing (icassp), pages 7074–7078. IEEE, 2020. find
  150. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
  151. E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
  152. Z. Hu, Z. Yang, X. Liang, R. Salakhutdinov, and E. P. Xing. Toward controlled generation of text. In International conference on machine learning, pages 1587–1596. PMLR, 2017.
  153. H. Huang, Z. Li, L. Wang, S. Chen, B. Dong, and X. Zhou. Feature space singularity for out-of-distribution detection. arXiv preprint arXiv:2011.14654, 2020. — [arXiv:2011.14654](https://arxiv.org/abs/2011.14654) find
  154. P.-S. Huang, R. Stanforth, J. Welbl, C. Dyer, D. Yogatama, S. Gowal, K. Dvijotham, and P. Kohli. Achieving verified robustness to symbol substitutions via interval bound propagation. arXiv preprint arXiv:1909.01492, 2019.
  155. W. Huang, Y. Sun, X. Zhao, J. Sharp, W. Ruan, J. Meng, and X. Huang. Coverage-guided testing for recurrent neural networks. IEEE Transactions on Reliability, 71(3):1191–1206, 2021. find
  156. W. Huang, X. Zhao, A. Banks, V. Cox, and X. Huang. Hierarchical distribution-aware testing of deep learning. arXiv preprint arXiv:2205.08589, 2022. — [arXiv:2205.08589](https://arxiv.org/abs/2205.08589) find
  157. W. Huang, X. Zhao, G. Jin, and X. Huang. Safari: Versatile and efficient evaluations for robustness of interpretability. arXiv preprint arXiv:2208.09418, 2022. — [arXiv:2208.09418](https://arxiv.org/abs/2208.09418) find
  158. X. Huang, M. Alzantot, and M. Srivastava. Neuroninspect: Detecting backdoors in neural networks via output explanations. arXiv preprint arXiv:1911.07399, 2019. — [arXiv:1911.07399](https://arxiv.org/abs/1911.07399) find
  159. X. Huang, G. Jin, and W. Ruan. Machine learning basics. In Machine Learning Safety, pages 3–13. Springer, 2012. find
  160. X. Huang, D. Kroening, W. Ruan, J. Sharp, Y. Sun, E. Thamo, M. Wu, and X. Yi. A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability. Computer Science Review, 37:100270, 2020. find
  161. X. Huang, M. Kwiatkowska, S. Wang, and M. Wu. Safety verification of deep neural networks. In R. Majumdar and V. Kuncak, editors, Computer Aided Verification - 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I, volume 10426 of Lecture Notes in Comput… find
  162. X. Huang, W. Ruan, Q. Tang, and X. Zhao. Bridging formal methods and machine learning with global optimisation. In Formal Methods and Software Engineering: 23rd International Conference on Formal Engineering Methods, ICFEM 2022, Madrid, Spain, October 24–27, 2022, Proceedings, page 1–19, B… find
  163. A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019. find
  164. A. Ivankay, I. Girardi, C. Marchiori, and P. Frossard. Fooling explanations in text classifiers. arXiv preprint arXiv:2206.03178, 2022. — [arXiv:2206.03178](https://arxiv.org/abs/2206.03178) find
  165. M. Iyyer, J. Wieting, K. Gimpel, and L. Zettlemoyer. Adversarial example generation with syntactically controlled paraphrase networks. arXiv preprint arXiv:1804.06059, 2018.
  166. A. Jaiswal, A. R. Babu, M. Z. Zadeh, D. Banerjee, and F. Makedon. A survey on contrastive self-supervised learning. Technologies, 9(1):2, 2020. find
  167. M. Jang and T. Lukasiewicz. Consistency analysis of chatgpt. arXiv preprint arXiv:2303.06273, 2023. find
  168. N. Jansen, B. Könighofer, J. Junges, A. Serban, and R. Bloem. Safe reinforcement learning using probabilistic shields. Dagstuhl: Schloss Dagstuhl, 2020. find
  169. N. Jansen, B. Könighofer, S. Junges, and R. Bloem. Shielded decision-making in mdps. arXiv preprint arXiv:1807.06096, 2018. — [arXiv:1807.06096](https://arxiv.org/abs/1807.06096) find
  170. Y. Ji, Y. Gong, Y. Peng, C. Ni, P. Sun, D. Pan, B. Ma, and X. Li. Exploring chatgpt’s ability to rank content: A preliminary study on consistency with human preferences, 2023. find
  171. R. Jia and P. Liang. Adversarial examples for evaluating reading comprehension systems. arXiv preprint arXiv:1707.07328, 2017.
  172. R. Jia, A. Raghunathan, K. Göksel, and P. Liang. Certified robustness to adversarial word substitutions. arXiv preprint arXiv:1909.00986, 2019.
  173. A. Q. Jiang, S. Welleck, J. P. Zhou, W. Li, J. Liu, M. Jamnik, T. Lacroix, Y. Wu, and G. Lample. Draft, sketch, and prove: Guiding formal theorem provers with informal proofs. arXiv preprint arXiv:2210.12283, 2022.
  174. W. Jiao, W. Wang, J.-t. Huang, X. Wang, and Z. Tu. Is chatgpt a good translator? a preliminary study. arXiv preprint arXiv:2301.08745, 2023.
  175. D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018–8025, 2020.
  176. K. S. Kalyan, A. Rajasekharan, and S. Sangeetha. Ammus: A survey of transformer-based pretrained models in natural language processing. arXiv preprint arXiv:2108.05542, 2021. — [arXiv:2108.05542](https://arxiv.org/abs/2108.05542) find
  177. S. Kambhampati. Changing the nature of ai research. Commun. ACM, 65(9):8–9, aug 2022. find
  178. R. Kande, H. Pearce, B. Tan, B. Dolan-Gavitt, S. Thakur, R. Karri, and J. Rajendran. Llm-assisted generation of hardware assertions. CoRR, abs/2306.14027, 2023. — [arXiv:2306.14027](https://arxiv.org/abs/2306.14027) find
  179. D. Kang, X. Li, I. Stoica, C. Guestrin, M. Zaharia, and T. Hashimoto. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. arXiv preprint arXiv:2302.05733, 2023. find
  180. Y. Kang, Q. Zhang, and R. Roth. The ethics of ai-generated maps: A study of dalle 2 and implications for cartography. arXiv preprint arXiv:2304.10743, 2023. — [arXiv:2304.10743](https://arxiv.org/abs/2304.10743) find
  181. J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
  182. D. M. Katz, M. J. Bommarito, S. Gao, and P. Arredondo. Gpt-4 passes the bar exam. Available at SSRN 4389233, 2023. find
  183. R. Khoury, A. R. Avila, J. Brunelle, and B. M. Camara. How secure is code generated by chatgpt? arXiv preprint arXiv:2304.09655, 2023. — [arXiv:2304.09655](https://arxiv.org/abs/2304.09655) find
  184. Y.-M. Kim. Data and fair use. Korea Copyright Commission, 141:5–53, 03 2023. find
  185. C.-Y. Ko, Z. Lyu, L. Weng, L. Daniel, N. Wong, and D. Lin. Popqorn: Quantifying robustness of recurrent neural networks. In International Conference on Machine Learning, pages 3468–3477. PMLR, 2019. find
  186. J. Y. Koh, D. Fried, and R. Salakhutdinov. Generating images with multimodal language models. arXiv preprint arXiv:2305.17216, 2023. — [arXiv:2305.17216](https://arxiv.org/abs/2305.17216) find
  187. V. Kuleshov, S. Thakoor, T. Lau, and S. Ermon. Adversarial examples for natural language classification problems. arXiv preprint, 2018. find
  188. A. Kumar, K. Ahuja, R. Vadapalli, and P. Talukdar. Syntax-guided controlled generation of paraphrases. Transactions of the Association for Computational Linguistics, 8:330–345, 2020.
  189. K. Kurita, P. Michel, and G. Neubig. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2793–2806, 2020.
  190. E. La Malfa, M. Wu, L. Laurenti, B. Wang, A. Hartshorn, and M. Kwiatkowska. Assessing robustness of text classification through maximal safe radius computation. arXiv preprint arXiv:2010.02004, 2020. find
  191. M. Lam, R. Sethi, J. D. Ullman, and A. Aho. Compilers: principles, techniques, and tools. Pearson Education, 2006. find
  192. N. Lambert, L. Castricato, L. von Werra, and A. Havrilla. Illustrating reinforcement learning from human feedback (rlhf). Hugging Face Blog, 2022. https://huggingface.co/blog/rlhf. ↗
  193. Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942, 2019.
  194. C. Lee, K. Cho, and W. Kang. Mixout: Effective regularization to finetune large-scale pretrained language models. arXiv preprint arXiv:1909.11299, 2019. find
  195. J. Y. Lee. Can an artificial intelligence chatbot be the author of a scholarly article? journal of Educational Evaluation for Health Professions, 20, 2023. find
  196. K. Lee, H. Liu, M. Ryu, O. Watkins, Y. Du, C. Boutilier, P. Abbeel, M. Ghavamzadeh, and S. S. Gu. Aligning text-to-image models using human feedback. arXiv preprint arXiv:2302.12192, 2023. — [arXiv:2302.12192](https://arxiv.org/abs/2302.12192) find
  197. N. Lee, Y. Bang, A. Madotto, and P. Fung. Misinformation has high perplexity. arXiv preprint arXiv:2006.04666, 2020. — [arXiv:2006.04666](https://arxiv.org/abs/2006.04666) find
  198. P. Lee. Learning from tay’s introduction. https://blogs.microsoft.com/blog/2016/03/25/learning-tays-introduction/, 25,05.2016. Accessed: 2023-08-20. ↗
  199. Y. Lei, Y. Cao, D. Li, T. Zhou, M. Fang, and M. Pechenizkiy. Phrase-level textual adversarial attack with label preservation. arXiv preprint arXiv:2205.10710, 2022. find
  200. D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668, 2020. find
  201. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Lingui…
  202. H. Li, D. Guo, W. Fan, M. Xu, and Y. Song. Multi-step jailbreaking privacy attacks on chatgpt. arXiv preprint arXiv:2304.05197, 2023.
  203. J. Li, X. Cheng, W. X. Zhao, J.-Y. Nie, and J.-R. Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. arXiv e-prints, pages arXiv–2305, 2023.
  204. J. Li, S. Ji, T. Du, B. Li, and T. Wang. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271, 2018.
  205. J. Li, T. Tang, W. X. Zhao, J.-Y. Nie, and J.-R. Wen. Pretrained language models for text generation: A survey. arXiv preprint arXiv:2201.05273, 2022. — [arXiv:2201.05273](https://arxiv.org/abs/2201.05273) find
  206. S. Li, H. Liu, T. Dong, B. Z. H. Zhao, M. Xue, H. Zhu, and J. Lu. Hidden backdoors in human-centric language models. In CCS ’21: 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual Event, Republic of Korea, November 15 - 19, 2021, pages 3123–3140. ACM, 2021. find
  207. X. Li, J. Li, X. Sun, C. Fan, T. Zhang, F. Wu, Y. Meng, and J. Zhang. kfolden: k-fold ensemble for out-of-distribution detection-fold ensemble for out-of-distribution detection. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3102–3115, 2021. find
  208. Y. Li, L. Ding, and X. Gao. On the decision boundary of deep neural networks. arXiv preprint arXiv:1808.05385, 2018. — [arXiv:1808.05385](https://arxiv.org/abs/1808.05385) find
  209. B. Liang, H. Li, M. Su, P. Bian, X. Li, and W. Shi. Deep text classification can be fooled. arXiv preprint arXiv:1704.08006, 2017. — [arXiv:1704.08006](https://arxiv.org/abs/1704.08006) find
  210. S. Liang, Y. Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In 6th International Conference on Learning Representations, ICLR 2018, 2018. find
  211. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In Computer Vision– ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  212. Z. Lin, P. Xu, G. I. Winata, F. B. Siddique, Z. Liu, J. Shin, and P. Fung. Caire: An empathetic neural chatbot. arXiv preprint arXiv:1907.12108, 2019. — [arXiv:1907.12108](https://arxiv.org/abs/1907.12108) find
  213. C. Liu, T. Arnon, C. Lazarus, C. Strong, C. Barrett, M. J. Kochenderfer, et al. Algorithms for verifying deep neural networks. Foundations and Trends® in Optimization, 4(3-4):244–404, 2021. find
  214. H. Liu, R. Ning, Z. Teng, J. Liu, Q. Zhou, and Y. Zhang. Evaluating the logical reasoning ability of chatgpt and gpt-4. arXiv preprint arXiv:2304.03439, 2023.
  215. J. Liu, C. S. Xia, Y. Wang, and L. Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. arXiv preprint arXiv:2305.01210, 2023.
  216. W. Liu, X. Wang, J. Owens, and Y. Li. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464–21475, 2020. find
  217. X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, and J. Tang. Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering, 35(1):857–876, 2021. find
  218. Y. Liu, T. Han, S. Ma, J. Zhang, Y. Yang, J. Tian, H. He, A. Li, M. He, Z. Liu, et al. Summary of chatgpt/gpt-4 research and perspective towards the future of large language models. arXiv preprint arXiv:2304.01852, 2023.
  219. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
  220. Z. Liu, Y. Wang, K. Han, W. Zhang, S. Ma, and W. Gao. Post-training quantization for vision transformer. Advances in Neural Information Processing Systems, 34:28092–28103, 2021. find
  221. Z. Liu, X. Yu, L. Zhang, Z. Wu, C. Cao, H. Dai, L. Zhao, W. Liu, D. Shen, Q. Li, et al. Deid-gpt: Zero-shot medical text de-identification by gpt-4. arXiv preprint arXiv:2303.11032, 2023. — [arXiv:2303.11032](https://arxiv.org/abs/2303.11032) find
  222. R. Lou, K. Zhang, and W. Yin. Is prompt all you need? no. a comprehensive and broader view of instruction learning. arXiv preprint arXiv:2303.10475, 2023. — [arXiv:2303.10475](https://arxiv.org/abs/2303.10475) find
  223. N. Madaan, I. Padhi, N. Panwar, and D. Saha. Generate your counterfactuals: Towards controlled counterfactual generation for text. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13516–13524, 2021. find
  224. A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.
  225. K. Malinka, M. Peresíni, A. Firc, O. Hujnák, and F. Janus. On the educational impact of chatgpt: Is artificial intelligence ready to obtain a university degree? In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, pages 47–53, 2023. find
  226. Z. Manna and A. Pnueli. The temporal logic of reactive and concurrent systems: Specification. Springer Science & Business Media, 2012. find
  227. N. Maus, P. Chao, E. Wong, and J. Gardner. Adversarial prompting for black box foundation models. arXiv preprint arXiv:2302.04237, 2023. — [arXiv:2302.04237](https://arxiv.org/abs/2302.04237) find
  228. W. McCune. Prover9 and mace4. https://www.cs.unm.edu/~mccune/prover9/, 2005. Accessed: 2023-08-20. ↗
  229. Y. Mehdi. Announcing the next wave of ai innovation with microsoft bing and edge, May 2023. find
  230. S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837, 2022.
  231. M. Mirman, T. Gehr, and M. Vechev. Differentiable abstract interpretation for provably robust neural networks. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 3578–3586. PMLR, 10–15 Ju… find
  232. S. Mitrović, D. Andreoletti, and O. Ayoub. Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt-generated text, 2023. find
  233. J. Monteiro, I. Albuquerque, Z. Akhtar, and T. H. Falk. Generalizable adversarial examples detection based on bi-model decision mismatch. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pages 2839– 2844. IEEE, 2019. find
  234. M. Nagel, R. A. Amjad, M. Van Baalen, C. Louizos, and T. Blankevoort. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning, pages 7197–7206. PMLR, 2020. find
  235. B. Nelson, M. Barreno, F. J. Chi, A. D. Joseph, B. I. P. Rubinstein, U. Saini, C. Sutton, J. D. Tygar, and K. Xia. Exploiting machine learning to subvert your spam filter. In Proceedings of the 1st Usenix Workshop on Large-Scale Exploits and Emergent Threats, LEET’08, USA, 2008. USENIX Association. find
  236. T. H. News. Wormgpt: New ai tool allows cybercriminals to launch sophisticated cyber attacks. https://thehackernews.com/2023/07/ wormgpt-new-ai-tool-allows.html, 2023. Accessed: 2023-08-20. ↗
  237. A. Ni, S. Iyer, D. Radev, V. Stoyanov, W.-t. Yih, S. Wang, and X. V. Lin. Lever: Learning to verify language-to-code generation with execution. In International Conference on Machine Learning, pages 26106–26128. PMLR, 2023.
  238. A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2021. find
  239. Y. Nie, A. Williams, E. Dinan, M. Bansal, J. Weston, and D. Kiela. Adversarial nli: A new benchmark for natural language understanding. arXiv preprint arXiv:1910.14599, 2019.
  240. OpenAI. GPT-4 Technical Report. arXiv e-prints 2303.08774, 2023.
  241. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  242. S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu. Unifying large language models and knowledge graphs: A roadmap, 2023.
  243. G. Pang, C. Shen, L. Cao, and A. V. D. Hengel. Deep learning for anomaly detection: A review. ACM computing surveys (CSUR), 54(2):1–38, 2021. find
  244. G. Park, B. Park, S. J. Kwon, B. Kim, Y. Lee, and D. Lee. nuqmm: Quantized matmul for efficient inference of large-scale generative language models. arXiv preprint arXiv:2206.09557, 2022.
  245. D. Patterson, J. Gonzalez, U. Holzle, Q. Le, C. Liang, L.-M. Munguia, D. Rothchild, D. R. So, M. Texier, and J. Dean. The carbon footprint of machine learning training will plateau, then shrink. Computer, 55(7):18–28, 2022. find
  246. H. Pearce, B. Tan, B. Ahmad, R. Karri, and B. Dolan-Gavitt. Examining zero-shot vulnerability repair with large language models. In 2023 IEEE Symposium on Security and Privacy (SP), pages 2339–2356. IEEE, 2023. find
  247. A. Pegoraro, K. Kumari, H. Fereidooni, and A.-R. Sadeghi. To chatgpt, or not to chatgpt: That is the question! arXiv preprint arXiv:2304.01487, 2023. — [arXiv:2304.01487](https://arxiv.org/abs/2304.01487) find
  248. B. Peng, C. Li, P. He, M. Galley, and J. Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023.
  249. J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014.
  250. F. Perez and I. Ribeiro. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527, 2022.
  251. A. Podolskiy, D. Lipin, A. Bout, E. Artemova, and I. Piontkovskaya. Revisiting mahalanobis distance for transformer-based out-of-domain detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13675–13682, 2021. find
  252. Y. Qi, X. Zhao, and X. Huang. safety analysis in the era of large language models: a case study of stpa using chatgpt. arXiv preprint arXiv:2304.01246, 2023. — [arXiv:2304.01246](https://arxiv.org/abs/2304.01246) find
  253. A. Radford, R. Jozefowicz, and I. Sutskever. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444, 2017.
  254. A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, et al. Improving language understanding by generative pre-training. OpenAI, 2018. find
  255. J. W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021. find
  256. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020.
  257. R. Ramamurthy, P. Ammanabrolu, K. Brantley, J. Hessel, R. Sifa, C. Bauckhage, H. Hajishirzi, and Y. Choi. Is reinforcement learning (not) for natural language processing?: Benchmarks, baselines, and building blocks for natural language policy optimization. arXiv preprint arXiv:2210.01241, 2022.
  258. A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
  259. A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pages 8821–8831. PMLR, 2021.
  260. M. V. Reiss. Testing the reliability of chatgpt for text annotation and classification: A cautionary remark. arXiv preprint arXiv:2304.11085, 2023. — [arXiv:2304.11085](https://arxiv.org/abs/2304.11085) find
  261. J. Ren, P. J. Liu, E. Fertig, J. Snoek, R. Poplin, M. Depristo, J. Dillon, and B. Lakshminarayanan. Likelihood ratios for out-of-distribution detection. Advances in neural information processing systems, 32, 2019. find
  262. S. Ren, Y. Deng, K. He, and W. Che. Generating natural language adversarial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1085–1097, 2019.
  263. X. Ren, P. Zhou, X. Meng, X. Huang, Y. Wang, W. Wang, P. Li, X. Zhang, A. Podolskiy, G. Arshinov, et al. Pangu-σ : Towards trillion parameter language model with sparse heterogeneous computing. arXiv preprint arXiv:2303.10845, 2023.
  264. M. T. Ribeiro, S. Singh, and C. Guestrin. "why should I trust you?": Explaining the predictions of any classifier. In HLT-NAACL Demos, 2016.
  265. J. T. Rolfe. Discrete variational autoencoders. arXiv preprint arXiv:1609.02200, 2016. — [arXiv:1609.02200](https://arxiv.org/abs/1609.02200) find
  266. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022.
  267. W. Ruan, X. Huang, and M. Kwiatkowska. Reachability analysis of deep neural networks with provable guarantees. In IJCAI2018, pages 2651–2659, 2018. find
  268. W. Ruan, M. Wu, Y. Sun, X. Huang, D. Kroening, and M. Kwiatkowska. Global robustness evaluation of deep neural networks with provable guarantees for the hamming distance. In IJCAI2019, pages 5944–5952, 2019. find
  269. S. Ruder, M. E. Peters, S. Swayamdipta, and T. Wolf. Transfer learning in natural language processing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials, pages 15–18, 2019.
  270. B. Rueckauer, I.-A. Lungu, Y. Hu, M. Pfeiffer, and S.-C. Liu. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in neuroscience, 11:682, 2017. find
  271. J. Rutinowski, S. Franke, J. Endendyk, I. Dormuth, and M. Pauly. The self-perception and political biases of chatgpt. arXiv preprint arXiv:2304.07333, 2023. — [arXiv:2304.07333](https://arxiv.org/abs/2304.07333) find
  272. W. Ryou, J. Chen, M. Balunovic, G. Singh, A. Dan, and M. Vechev. Scalable polyhedral verification of recurrent neural networks. In International Conference on Computer Aided Verification, pages 225–248. Springer, 2021. find
  273. C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494, 2022.
  274. S. Samanta and S. Mehta. Towards crafting text adversarial samples. arXiv preprint arXiv:1707.02812, 2017. find
  275. G. Sandoval, H. Pearce, T. Nys, R. Karri, S. Garg, and B. Dolan-Gavitt. Lost at c: A user study on the security implications of large language model code assistants. arXiv preprint arXiv:2208.09727, 2023. — [arXiv:2208.09727](https://arxiv.org/abs/2208.09727) find
  276. T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100, 2022. find
  277. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. find
  278. U. Senate. Senate judiciary subcommittee hearing on oversight of ai. https://techpolicy.press/ transcript-senate-judiciary-subcommittee-hearing-on-oversight-of-ai/, 2023. Accessed: 2023-08-20. ↗
  279. S. A. Seshia, D. Sadigh, and S. S. Sastry. Towards verified artificial intelligence. arXiv preprint arXiv:1606.08514, 2016. — [arXiv:1606.08514](https://arxiv.org/abs/1606.08514) find
  280. M. Shanahan. Talking about large language models. arXiv:2212.03551, 2022. arXiv preprint — [arXiv:2212.03551](https://arxiv.org/abs/2212.03551) find
  281. L. Shen, S. Ji, X. Zhang, J. Li, J. Chen, J. Shi, C. Fang, J. Yin, and T. Wang. Backdoor pre-trained models can transfer to all. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 3141–3158, 2021. find
  282. X. Shen, Z. Chen, M. Backes, and Y. Zhang. In chatgpt we trust? measuring and characterizing the reliability of chatgpt. arXiv preprint arXiv:2304.08979, 2023. — [arXiv:2304.08979](https://arxiv.org/abs/2304.08979) find
  283. Y. Shen, Y.-C. Hsu, A. Ray, and H. Jin. Enhancing the generalization for intent classification and out-of-domain detection in slu. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Vo… find
  284. Z. Shi, H. Zhang, K.-W. Chang, M. Huang, and C.-J. Hsieh. Robustness verification for transformers. In International Conference on Learning Representations, 2019. find
  285. K. Shuster, M. Komeili, L. Adolphs, S. Roller, A. Szlam, and J. Weston. Language models that seek for knowledge: Modular search & generation for dialogue and prompt completion. arXiv preprint arXiv:2203.13224, 2022.
  286. K. Shuster, S. Poff, M. Chen, D. Kiela, and J. Weston. Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567, 2021.
  287. A. Sinha, H. Namkoong, R. Volpi, and J. Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571, 2017. — [arXiv:1710.10571](https://arxiv.org/abs/1710.10571) find
  288. L. Smith and Y. Gal. Understanding measures of uncertainty for adversarial example detection. arXiv preprint arXiv:1803.08533, 2018. — [arXiv:1803.08533](https://arxiv.org/abs/1803.08533) find
  289. S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti, et al. Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model. arXiv preprint arXiv:2201.11990, 2022.
  290. D. Sobania, M. Briesch, C. Hanna, and J. Petke. An analysis of the automatic bug fixing performance of chatgpt. arXiv preprint arXiv:2301.08653, 2023. — [arXiv:2301.08653](https://arxiv.org/abs/2301.08653) find
  291. S. Soltan, S. Ananthakrishnan, J. FitzGerald, R. Gupta, W. Hamza, H. Khan, C. Peris, S. Rawls, A. Rosenbaum, A. Rumshisky, et al. Alexatm 20b: Few-shot learning using a large-scale multilingual seq2seq model. arXiv preprint arXiv:2208.01448, 2022.
  292. L. Struppek, D. Hintersdorf, and K. Kersting. Rickrolling the artist: Injecting invisible backdoors into text-guided image generation models. arXiv preprint arXiv:2211.02408, 2022. — [arXiv:2211.02408](https://arxiv.org/abs/2211.02408) find
  293. H. Sun, Z. Zhang, J. Deng, J. Cheng, and M. Huang. Safety assessment of chinese large language models. arXiv preprint arXiv:2304.10436, 2023.
  294. Y. Sun, X. Huang, D. Kroening, J. Sharp, M. Hill, and R. Ashmore. Testing deep neural networks. arXiv preprint arXiv:1803.04792, 2018. — [arXiv:1803.04792](https://arxiv.org/abs/1803.04792) find
  295. Y. Sun, X. Huang, D. Kroening, J. Sharp, M. Hill, and R. Ashmore. Structural test coverage criteria for deep neural networks. ACM Trans. Embed. Comput. Syst., 18(5s), Oct. 2019. find
  296. Y. Sun, S. Wang, S. Feng, S. Ding, C. Pang, J. Shang, J. Liu, X. Chen, Y. Zhao, Y. Lu, et al. Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv:2107.02137, 2021.
  297. Y. Sun, M. Wu, W. Ruan, X. Huang, M. Kwiatkowska, and D. Kroening. Concolic testing for deep neural networks. In ASE2018, 2018. find
  298. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.
  299. L. Tanguy, N. Tulechki, A. Urieli, E. Hermann, and C. Raynal. Natural language processing for aviation safety reports: From classification to interactive analysis. Computers in Industry, 78:80–95, 2016. find
  300. R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto. Stanford alpaca: An instruction-following llama model, 2023. find
  301. R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, and R. Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022.
  302. A. Tejankar, M. Sanjabi, Q. Wang, S. Wang, H. Firooz, H. Pirsiavash, and L. Tan. Defending against patch-based backdoor attacks on self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12239–12249, 2023. find
  303. S. Thakur, B. Ahmad, Z. Fan, H. Pearce, B. Tan, R. Karri, B. Dolan-Gavitt, and S. Garg. Benchmarking large language models for automated verilog rtl code generation. In 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1–6. IEEE, 2023. find
  304. R. Thoppilan, D. De Freitas, J. Hall, N. Shazeer, A. Kulshreshtha, H.-T. Cheng, A. Jin, T. Bos, L. Baker, Y. Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022.
  305. J. Thorne, A. Vlachos, C. Christodoulopoulos, and A. Mittal. Fever: A large-scale dataset for fact extraction and verification. In 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018, pages 809–819. Association f…
  306. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
  307. A. S. Tulshan and S. N. Dhage. Survey on virtual assistant: Google assistant, siri, cortana, alexa. In Advances in Signal Processing and Intelligent Recognition Systems: 4th International Symposium SIRS 2018, Bangalore, India, September 19–22, 2018, Revised Selected Papers 4, pages 190–201. Springer… find
  308. A. Uchendu, J. Lee, H. Shen, T. Le, T. K. Huang, and D. Lee. Understanding individual and team-based human factors in detecting deepfake texts. CoRR, abs/2304.01002, 2023. — [arXiv:2304.01002](https://arxiv.org/abs/2304.01002) find
  309. M. Y. Vardi and P. Wolper. An automata-theoretic approach to automatic program verification. In 1st Symposium in Logic in Computer Science (LICS). IEEE Computer Society, 1986. find
  310. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  311. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Cu…
  312. M. Wallace, R. Khandelwal, and B. Tang. Does ibp scale? arXiv preprint, 2022. find
  313. B. Wang, C. Xu, S. Wang, Z. Gan, Y. Cheng, J. Gao, A. H. Awadallah, and B. Li. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2111.02840, 2021. find
  314. F. Wang, P. Xu, W. Ruan, and X. Huang. Towards verifying the geometric robustness of large-scale neural networks. arXiv preprint arXiv:2301.12456, 2023. — [arXiv:2301.12456](https://arxiv.org/abs/2301.12456) find
  315. G. Wang, Y. Lin, and W. Yi. Kernel fusion: An effective method for better power efficiency on multithreaded gpu. In 2010 IEEE/ACM Int’l Conference on Green Computing and Communications & Int’l Conference on Cyber, Physical and Social Computing, pages 344–350. IEEE, 2010. find
  316. J. Wang, X. Hu, W. Hou, H. Chen, R. Zheng, Y. Wang, L. Yang, H. Huang, W. Ye, X. Geng, et al. On the robustness of chatgpt: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2302.12095, 2023.
  317. J. Wang, X. Hu, W. Hou, H. Chen, R. Zheng, Y. Wang, L. Yang, H. Huang, W. Ye, X. Geng, B. Jiao, Y. Zhang, and X. Xie. On the Robustness of Chat-GPT: An Adversarial and Out-of-distribution Perspective. arXiv e-prints, page arXiv:2302.12095, Feb. 2023.
  318. W. Wang, P. Tang, J. Lou, and L. Xiong. Certified robustness to word substitution attack with differential privacy. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1102–1112, 2021. find
  319. X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023.
  320. Y. Wang and M. Bansal. Robust machine comprehension models via adversarial training. arXiv preprint arXiv:1804.06473, 2018. find
  321. J. Wei, S. Kim, H. Jung, and Y.-H. Kim. Leveraging large language models to power chatbots for collecting user self-reported data. arXiv preprint arXiv:2301.05843, 2023. — [arXiv:2301.05843](https://arxiv.org/abs/2301.05843) find
  322. J. Wei, X. Wang, D. Schuurmans, M. Bosma, brian ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022. — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  323. T.-W. Weng, H. Zhang, P.-Y. Chen, J. Yi, D. Su, Y. Gao, C.-J. Hsieh, and L. Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. arXiv preprint arXiv:1801.10578, 2018. — [arXiv:1801.10578](https://arxiv.org/abs/1801.10578) find
  324. Y. Weng, M. Zhu, S. He, K. Liu, and J. Zhao. Large language models are reasoners with self-verification. arXiv preprint arXiv:2212.09561, 2022.
  325. Y. Weng, M. Zhu, F. Xia, B. Li, S. He, K. Liu, and J. Zhao. Neural comprehension: Language models with compiled neural networks. arXiv preprint arXiv:2304.01665, 2023. — [arXiv:2304.01665](https://arxiv.org/abs/2304.01665) find
  326. M. Wicker, X. Huang, and M. Kwiatkowska. Feature-guided black-box safety testing of deep neural networks. In Tools and Algorithms for the Construction and Analysis of Systems: 24th International Conference, TACAS 2018, Held as Part of the European Joint Conferences on Theory and Practice of Software… find
  327. Y. Wolf, N. Wies, Y. Levine, and A. Shashua. Fundamental limitations of alignment in large language models. arXiv preprint arXiv:2304.11082, 2023.
  328. E. Wong, L. Rice, and J. Z. Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.03994, 2020. — [arXiv:2001.03994](https://arxiv.org/abs/2001.03994) find
  329. D. Wu, G. Jin, H. Yu, X. Yi, and X. Huang. Optimising event-driven spiking neural network with regularisation and cutoff. arXiv preprint arXiv:2301.09522, 2023. — [arXiv:2301.09522](https://arxiv.org/abs/2301.09522) find
  330. D. Wu, X. Yi, and X. Huang. A little energy goes a long way: Build an energy-efficient, accurate spiking neural network from convolutional neural network. Frontiers in neuroscience, 16, 2022. find
  331. H. Wu, W. Wang, Y. Wan, W. Jiao, and M. Lyu. Chatgpt or grammarly? evaluating chatgpt on grammatical error correction benchmark. arXiv preprint arXiv:2303.13648, 2023.
  332. M. Wu, A. Waheed, C. Zhang, M. Abdul-Mageed, and A. F. Aji. Lamini-lm: A diverse herd of distilled models from large-scale instructions. arXiv preprint arXiv:2304.14402, 2023.
  333. M. Wu, M. Wicker, W. Ruan, X. Huang, and M. Kwiatkowska. A game-based approximate verification of deep neural networks with provable guarantees. Theoretical Computer Science, 807:298–329, 2020. find
  334. S. Wu, O. Irsoy, S. Lu, V. Dabravolski, M. Dredze, S. Gehrmann, P. Kambadur, D. Rosenberg, and G. Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023.
  335. X. Wu, K. Sun, F. Zhu, R. Zhao, and H. Li. Better aligning text-to-image models with human preference. arXiv preprint arXiv:2303.14420, 2023. find
  336. Y. Wu, A. Q. Jiang, W. Li, M. N. Rabe, C. E. Staats, M. Jamnik, and C. Szegedy. Autoformalization with large language models. In A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, editors, Advances in Neural Information Processing Systems, 2022.
  337. F. Xu, H. Uszkoreit, Y. Du, W. Fan, D. Zhao, and J. Zhu. Explainable ai: A brief survey on history, research areas, approaches and challenges. In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9–14, 2019, Proceedings, Part II… find
  338. H. Xu, K. He, Y. Yan, S. Liu, Z. Liu, and W. Xu. A deep generative distance-based classifier for out-of-domain detection with mahalanobis space. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1452–1460, 2020. find
  339. H. Xu, Y. Ma, H.-C. Liu, D. Deb, H. Liu, J.-L. Tang, and A. K. Jain. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing, 17:151–178, 2020. find
  340. J. Xu, X. Liu, Y. Wu, Y. Tong, Q. Li, M. Ding, J. Tang, and Y. Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. arXiv preprint arXiv:2304.05977, 2023. find
  341. P. Xu, W. Ruan, and X. Huang. Quantifying safety risks of deep neural networks. Complex & Intelligent Systems, pages 1–18, 2022. find
  342. Yandex. Yandex/yalm-100b: Pretrained language model with 100b parameters. https://github.com/yandex/YaLM-100B. Accessed: 2023-08-20. ↗
  343. J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, B. Yin, and X. Hu. Harnessing the power of llms in practice: A survey on chatgpt and beyond. arXiv preprint arXiv:2304.13712, 2023.
  344. J. Yang, K. Zhou, Y. Li, and Z. Liu. Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334, 2021. — [arXiv:2110.11334](https://arxiv.org/abs/2110.11334) find
  345. W. Yang, L. Li, Z. Zhang, X. Ren, X. Sun, and B. He. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech…
  346. Z. Yang. Chinese tech giant baidu just released its answer to chatgpt, Mar 2023. find
  347. Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems, 32, 2019.
  348. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023.
  349. Z. Yao, R. Yazdani Aminabadi, M. Zhang, X. Wu, C. Li, and Y. He. Zeroquant: Efficient and affordable post-training quantization for large-scale transformers. Advances in Neural Information Processing Systems, 35:27168–27183, 2022.
  350. M. Ye, C. Gong, and Q. Liu. Safer: A structure-free approach for certified robustness to adversarial word substitutions. arXiv preprint arXiv:2005.14424, 2020.
  351. X. Ye, S. Iyer, A. Celikyilmaz, V. Stoyanov, G. Durrett, and R. Pasunuru. Complementary explanations for effective in-context learning. arXiv preprint arXiv:2211.13892, 2022.
  352. E. Yilmaz and C. Toraman. D2u: Distance-to-uniform learning for out-of-scope detection. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2093–2108, 2022. find
  353. J. Yu, Y. Xu, J. Y. Koh, T. Luong, G. Baid, Z. Wang, V. Vasudevan, A. Ku, Y. Yang, B. K. Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789, 2022. — [arXiv:2206.10789](https://arxiv.org/abs/2206.10789) find
  354. J. Zeng, X. Zheng, J. Xu, L. Li, L. Yuan, and X. Huang. Certified robustness to text adversarial attacks by randomized [mask]. arXiv preprint arXiv:2105.03743, 2021.
  355. W. Zeng, X. Ren, T. Su, H. Wang, Y. Liao, Z. Wang, X. Jiang, Z. Yang, K. Wang, X. Zhang, et al. Pangu-α: Large-scale autoregressive pretrained chinese language models with auto-parallel computation. arXiv preprint arXiv:2104.12369, 2021.
  356. W. Zeng, X. Ren, T. Su, H. Wang, Y. Liao, Z. Wang, X. Jiang, Z. Yang, K. Wang, X. Zhang, et al. Pangu-α: Large-scale autoregressive pretrained chinese language models with auto-parallel computation. arXiv preprint arXiv:2104.12369, 2021.
  357. Z. Zeng, K. He, Y. Yan, Z. Liu, Y. Wu, H. Xu, H. Jiang, and W. Xu. Modeling discriminative representations for out-of-domain detection with supervised contrastive learning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Con… find
  358. C. Zhang, W. Ruan, F. Wang, P. Xu, G. Min, and X. Huang. Model-agnostic reachability analysis on deep neural networks. arXiv preprint arXiv:2304.00813, 2023. — [arXiv:2304.00813](https://arxiv.org/abs/2304.00813) find
  359. C. Zhang, W. Ruan, and P. Xu. Reachability analysis of neural network control systems. arXiv preprint arXiv:2301.12100, 2023. — [arXiv:2301.12100](https://arxiv.org/abs/2301.12100) find
  360. J. Zhang, Y. Zhao, M. Saleh, and P. Liu. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 11328–113…
  361. S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022.
  362. T. Zhang, F. Ladhak, E. Durmus, P. Liang, K. McKeown, and T. B. Hashimoto. Benchmarking large language models for news summarization. arXiv preprint arXiv:2301.13848, 2023.
  363. Y. Zhang, A. Albarghouthi, and L. D’Antoni. Certified robustness to programmable transformations in lstms. arXiv preprint arXiv:2102.07818, 2021. find
  364. R. Zhao, X. Li, Y. K. Chia, B. Ding, and L. Bing. Can chatgpt-like generative models guarantee factual accuracy? on the mistakes of new generation search engines. arXiv preprint arXiv:2304.11076, 2023. — [arXiv:2304.11076](https://arxiv.org/abs/2304.11076) find
  365. W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.
  366. X. Zhao, W. Huang, X. Huang, V. Robu, and D. Flynn. Baylime: Bayesian local interpretable model-agnostic explanations. In C. de Campos and M. H. Maathuis, editors, Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Re… find
  367. X. Zhao, W. Huang, S. Schewe, Y. Dong, and X. Huang. Detecting operational adversarial examples for reliable deep learning. In 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S), pages 5–6, 2021. find
  368. Z. Zhao, D. Dua, and S. Singh. Generating natural adversarial examples. arXiv preprint arXiv:1710.11342, 2017. find
  369. Q. Zhong, L. Ding, J. Liu, B. Du, and D. Tao. Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint arXiv:2302.10198, 2023. find
  370. C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang, K. Zhang, C. Ji, Q. Yan, L. He, et al. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt. arXiv preprint arXiv:2302.09419, 2023. find
  371. W. Zhou, F. Liu, and M. Chen. Contrastive out-of-distribution detection for pretrained transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021. find
  372. Y. Zhou, P. Liu, and X. Qiu. Knn-contrastive learning for out-of-domain intent classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5129–5141, 2022. find
  373. R.-J. Zhu, Q. Zhao, and J. K. Eshraghian. Spikegpt: Generative pre-trained language model with spiking neural networks. arXiv preprint arXiv:2302.13939, 2023. — [arXiv:2302.13939](https://arxiv.org/abs/2302.13939) find
  374. D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/huang-2023-survey-safety-trustworthiness-large-language/