📄 Introducing v0.5 of the AI Safety Benchmark from MLCommons

Paper · 2024 · Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed et al.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

🏛️ Institutions

🏢GoogleMountain View · United States430 works · 424,363 citations🎓Stanford UniversityStanford · United States434 works · 314,419 citations🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🏢MicrosoftRedmond · United States325 works · 142,931 citations🎓New York UniversityNew York · United States201 works · 100,180 citations🎓University of OxfordOxford · United Kingdom104 works · 157,183 citations🎓University of Massachusetts AmherstAmherst Center · United States106 works · 66,692 citations🎓University of TrentoTrento · Italy85 works · 15,524 citations🎓The Ohio State UniversityColumbus · United States82 works · 18,502 citations🎓University of California, Santa BarbaraSanta Barbara · United States57 works · 9,752 citations🔬German Research Centre for Artificial IntelligenceKaiserslautern · Germany77 works · 29,479 citations🎓University of ChicagoChicago · United States45 works · 50,617 citations🎓Bocconi UniversityMilan · Italy28 works · 3,077 citations🎓Polytechnique MontréalMontreal · Canada9 works · 27,467 citations🎓University of YorkYork · United Kingdom17 works · 6,570 citations🏢IntelSanta Clara · United States15 works · 41,257 citations🎓Illinois Institute of TechnologyChicago · United States12 works · 1,006 citations🎓Indian Institute of Technology DelhiNew Delhi · India22 works · 1,036 citations🔬Hessian Center for Artificial IntelligenceDarmstadt · Germany12 works · 529 citations🏢CohereSan Francisco · United States15 works · 1,223 citations🎓Iowa State UniversityAmes · United States8 works · 400 citations🏢SonyTokyo · Japan2 works · 1,722 citations🎓California Institute of TechnologyPasadena · United States8 works · 745 citations🎓Eindhoven University of TechnologyEindhoven · Netherlands8 works · 602 citations🏢Cerebras SystemsSunnyvale · United States4 works · 294 citations🏢QualcommSan Diego · United States6 works · 144 citations🔬Lawrence Livermore National LaboratoryLivermore · United States5 works · 456 citations🎓National UniversityManila · Philippines5 works · 353 citations🎓University of BathBath · United Kingdom5 works · 351 citations💡RAND CorporationSanta Monica · United States2 works · 38 citations

Also on the byline: Nike · Center for Security and Emerging Technology.

🧭 Fields, topics and concepts

Cite

Cite as Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Max Bartolo, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Simeon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert, Leon Derczynski, Debojyoti Dutta, Ian Eisenberg, James Ezick, Heather Frase, Brian Fuller, Ram Gandikota, Agasthya Gangavarapu, Ananya Gangavarapu, James Gealy, Rajat Ghosh, James Goel, Usman Gohar, Sujata Goswami, Scott A. Hale, Wiebke Hutiri, Joseph Marvin Imperial, Surgan Jandial, Nick Judd, Felix Juefei-Xu, Foutse Khomh, Bhavya Kailkhura, Hannah Rose Kirk, Kevin Klyman, Chris Knotz, Michael Kuchnik, Shachi H. Kumar, Srijan Kumar, Chris Lengerich, Bo Li, Zeyi Liao, Eileen Peters Long, Victor Lu, Sarah Luger, Yifan Mai, Priyanka Mary Mammen, Kelvin Manyeki, Sean McGregor, Virendra Mehta, Shafee Mohammed, Emanuel Moss, Lama Nachman, Dinesh Jinenhally Naganna, Amin Nikanjam, Besmira Nushi, Luis Oala, Iftach Orr, Alicia Parrish, Cigdem Patlak, William Pietri, Forough Poursabzi-Sangdeh, Eleonora Presani, Fabrizio Puletti, Paul Rottger, Saurav Sahay, Tim Santos, Nino Scherrer, Alice Schoenauer Sebag, Patrick Schramowski, Abolfazl Shahbazi, Vin Sharma, Xudong Shen, Vamsi Sistla, Leonard Tang, Davide Testuggine, Vithursan Thangarasa, Elizabeth Anne Watkins, Rebecca Weiss, Chris Welty, Tyler Wilbers, Adina Williams, Carole-Jean Wu, Poonam Yadav, Xianjun Yang, Yi Zeng, Wenhui Zhang, Fedor Zhdanov, Jiacheng Zhu, Percy Liang, Peter Mattson and Joaquin Vanschoren. 2024. Introducing v0.5 of the AI Safety Benchmark from MLCommons. arXiv:2404.12241.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2404.12241 · arXiv 2404.12241

@misc{vidgen2024introducing,
  title = {{Introducing v0.5 of the AI Safety Benchmark from MLCommons}},
  author = {Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Max Bartolo, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Simeon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert, Leon Derczynski, Debojyoti Dutta, Ian Eisenberg, James Ezick, Heather Frase, Brian Fuller, Ram Gandikota, Agasthya Gangavarapu, Ananya Gangavarapu, James Gealy, Rajat Ghosh, James Goel, Usman Gohar, Sujata Goswami, Scott A. Hale, Wiebke Hutiri, Joseph Marvin Imperial, Surgan Jandial, Nick Judd, Felix Juefei-Xu, Foutse Khomh, Bhavya Kailkhura, Hannah Rose Kirk, Kevin Klyman, Chris Knotz, Michael Kuchnik, Shachi H. Kumar, Srijan Kumar, Chris Lengerich, Bo Li, Zeyi Liao, Eileen Peters Long, Victor Lu, Sarah Luger, Yifan Mai, Priyanka Mary Mammen, Kelvin Manyeki, Sean McGregor, Virendra Mehta, Shafee Mohammed, Emanuel Moss, Lama Nachman, Dinesh Jinenhally Naganna, Amin Nikanjam, Besmira Nushi, Luis Oala, Iftach Orr, Alicia Parrish, Cigdem Patlak, William Pietri, Forough Poursabzi-Sangdeh, Eleonora Presani, Fabrizio Puletti, Paul Rottger, Saurav Sahay, Tim Santos, Nino Scherrer, Alice Schoenauer Sebag, Patrick Schramowski, Abolfazl Shahbazi, Vin Sharma, Xudong Shen, Vamsi Sistla, Leonard Tang, Davide Testuggine, Vithursan Thangarasa, Elizabeth Anne Watkins, Rebecca Weiss, Chris Welty, Tyler Wilbers, Adina Williams, Carole-Jean Wu, Poonam Yadav, Xianjun Yang, Yi Zeng, Wenhui Zhang, Fedor Zhdanov, Jiacheng Zhu, Percy Liang, Peter Mattson and Joaquin Vanschoren},
  year = {2024},
  doi = {10.48550/arxiv.2404.12241},
  eprint = {2404.12241},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.12241}
}

References

180 references, 44 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III au2, and Kate Crawford. Datasheets for datasets, 2021. (Cited on pages 2, 21) find
2. Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximilien Breughe, Mark Charlebois, William Chou, Ramesh Chukka, Cody Coleman, Sam Davis, Pan Deng, Greg Diamos, Jared Duke, Dave Fick, J. Scott Gardner, Itay Hubara, Sachin Idgun… find
3. S. McGregor. Open digital safety. Computer, 57(04):99–103, apr 2024. ISSN 1558-0814. doi: 10.1109/MC.2023.3315028. (Cited on pages 5) — [doi:10.1109/mc.2023.3315028](https://doi.org/10.1109/mc.2023.3315028) find
4. Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessic… — LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
5. Sayash Kapoor, Peter Henderson, and Arvind Narayanan. Promises and pitfalls of artificial intelligence for legal applications, 2024. (Cited on pages 5) https://huggingface.co/spaces/AI-Secure/llm-trustworthy-leaderboard https://livablesoftware.com/biases-llm-leaderboard/ 32 https://safetyprompts.com… ↗
6. Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. Financebench: A new benchmark for financial question answering. arXiv preprint arXiv:2311.11944, 2023. (Cited on pages 5, 27) — [arXiv:2311.11944](https://arxiv.org/abs/2311.11944) find
7. Mahyar Abbasian, Elahe Khatibi, Iman Azimi, David Oniani, Zahra Shakeri Hossein Abad, Alexander Thieme, Ram Sriram, Zhongqi Yang, Yanshan Wang, Bryant Lin, et al. Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative ai. npj Digital Medicine, 7(1):82, 2024… find
8. Jennifer Chubb, Sondess Missaoui, Shauna Concannon, Liam Maloney, and James Alfred Walker. Interactive storytelling for children: A case-study of design and development considerations for ethical conversational ai. International Journal of Child-Computer Interaction, 32:100403, 2022. ISSN 2212-8689.… find
9. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety, 2016. (Cited on pages 5)
10. Lindsay Sanneman and Julie A. Shah. The situation awareness framework for explainable ai (safe-ai) and human factors considerations for xai systems. International Journal of Human–Computer Interaction, 38(18-20):1772–1788, 2022. doi: 10.1080/104473 18.2022.2081282. URL https://doi.org/10.1080/104473… — [doi:10.1080/104473](https://doi.org/10.1080/104473) ↗
11. Rishi Bommasani et al. On the opportunities and risks of foundation models, 2022.
12. Krishnaram Kenthapadi, Himabindu Lakkaraju, and Nazneen Rajani. Generative ai meets responsible ai: Practical challenges and opportunities. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, page 5805–5806, New York, NY, USA, 2023. Association for Compu… — [doi:10.1145/3580305.3599557](https://doi.org/10.1145/3580305.3599557) find
13. Paul Röttger, Fabio Pernisi, Bertie Vidgen, and Dirk Hovy. Safetyprompts: a systematic review of open datasets for evaluating and improving large language model safety, 2024. (Cited on pages 5, 30)
14. Miles Brundage, Shahar Avin, Jack Clark, Helen Toner, Peter Eckersley, Ben Garfinkel, Allan Dafoe, Paul Scharre, Thomas Zeitzoff, Bobby Filar, Hyrum Anderson, Heather Roff, Gregory C. Allen, Jacob Steinhardt, Carrick Flynn, Seán Ó hÉigeartaigh, Simon Beard, Haydn Belfield, Sebastian Farquhar, Clare… find
15. Thomas G Dietterich and Eric J Horvitz. Rise of concerns about ai: reflections and directions. Communications of the ACM, 58(10):38–40, 2015. (Cited on pages 5) find
16. Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, and William Isaac. Sociotechnical safety evaluation of generative ai systems, 2023. (Cited on pages 26, 28,… find
17. Wiebke Hutiri, Oresiti Papakyriakopoulos, and Alice Xiang. Not my voice! a taxonomy of ethical and safety harms of speech generators, 2024. (Cited on pages 5) find
18. ISO/IEC/IEEE. Iso/iec/ieee 24748-7000:2022. systems and software engineering life cycle management part 7000: Standard model process for addressing ethical concerns during system design, 2024. URL https://www.iso.org/standard/84893.html. (Cited on pages 5) ↗
19. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric… find
20. Aarohi Srivastava et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models, 2022. (Cited on pages 5, 29, 30)
21. Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Benita Teufel, Marco Bellagente, Minguk Kang, Taesung Park, Jure Leskovec, Jun-Yan Zhu, Li Fei-Fei, Jiajun Wu, Stefano Ermon, and Percy Liang. Holistic evaluation of text-to-ima… find
22. Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models, 2023. (Cited on pages 5) find
23. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. (Cited o…
24. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — Llama 2: Open Foundation and Fine-Tuned Chat Models
25. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023. (Cited on pages 6) ↗
26. Yash Raj Shrestha, Georg von Krogh, and Stefan Feuerriegel. Building open-source ai. Nature Computational Science, 3(11):908–911, 2023. (Cited on pages 7) find
27. Huseyin A Inan, Osman Ramadan, Lukas Wutschitz, Daniel Jones, Victor Rühle, James Withers, and Robert Sim. Training data leakage analysis in language models. arXiv preprint arXiv:2101.05405, 2021. (Cited on pages 7) — [arXiv:2101.05405](https://arxiv.org/abs/2101.05405) find
28. Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. Benchmark probing: Investigating data leakage in large language models. In NeurIPS 2023 Workshop on Backdoors in Deep Learning - The Good, the Bad, and the Ugly, 2024. URL https://openreview.net/forum?id=a34bgvner1. ↗
29. Nishanth Chandran, Sunayana Sitaram, Divya Gupta, Rahul Sharma, Kashish Mittal, and Manohar Swaminathan. Private benchmarking to prevent contamination and improve comparative evaluation of llms, 2024. (Cited on pages 7) find
30. Bertie Vidgen, Nino Scherrer, Hannah Rose Kirk, Rebecca Qian, Anand Kannappan, Scott A. Hale, and Paul Röttger. Simplesafetytests: a test suite for identifying critical safety risks in large language models, 2024. (Cited on pages 8, 28, 49)
31. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: A test suite for identifying exaggerated safety behaviours in large language models, 2024. (Cited on pages 10, 26, 28)
32. Heather Hedden. The accidental taxonomist. Information Today, Inc, Medford, New Jersey, third edition edition, 2022. ISBN 978-1-57387-681-0 978-1-57387-682-7. (Cited on pages 10) find
33. Kevin Klyman. Acceptable use policies for foundation models, 2024. URL https://crfm.stanford.edu/2024/04/08/aups.html. (Cited on pages 12, 47) ↗
34. Seraphina Goldfarb-Tarrant, Eddie Ungless, Esma Balkir, and Su Lin Blodgett. This prompt is measuring\< mask\>: evaluating bias evaluation in language models. arXiv preprint arXiv:2305.12757, 2023. (Cited on pages 16) — [arXiv:2305.12757](https://arxiv.org/abs/2305.12757) find
35. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain,… — Discovering Language Model Behaviors with Model-Written Evaluations
36. Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, and Deep Ganguli. Evaluating and mitigating discrimination in language model decisions. arXiv preprint arXiv:2312.03689, 2023. find
37. Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. Safetybench: Evaluating the safety of large language models with multiple choice questions, 2023. (Cited on pages 16, 30, 48)
38. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R Bowman. Bbq: A hand-built bias benchmark for question answering. arXiv preprint arXiv:2110.08193, 2021. (Cited on pages 16)
39. Pouya Pezeshkpour and Estevam Hruschka. Large language models sensitivity to the order of options in multiple-choice questions. arXiv preprint arXiv:2308.11483, 2023. (Cited on pages 16)
40. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36, 2024. (Cited on pages 16) find
41. Michael Kuchnik, Virginia Smith, and George Amvrosiadis. Validating large language models with relm. In Sixth Conference on Machine Learning and Systems (MLSys 2023), June 2023. (Cited on pages 16, 27) find
42. Debora Nozza, Federico Bianchi, Dirk Hovy, et al. Honest: Measuring hurtful sentence completion in language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguis…
43. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462, 2020. (Cited on pages 17, 22)
44. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. The woman worked as a babysitter: On biases in language generation. arXiv preprint arXiv:1909.01326, 2019. (Cited on pages 17, 27)
45. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. Bold: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21. ACM, Mar… — [doi:10.1145/3442188.3445924](https://doi.org/10.1145/3442188.3445924) find
46. Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. Towards understanding and mitigating social biases in language models. In International Conference on Machine Learning, pages 6565–6576. PMLR, 2021. (Cited on pages 17)
47. Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston. Build it break it fix it for dialogue safety: Robustness from adversarial human attack. arXiv preprint arXiv:1908.06083, 2019. (Cited on pages 17)
48. Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators, 2023. find
49. Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, et al. A strongreject for empty jailbreaks. arXiv preprint arXiv:2402.10260, 2024. (Cited on pages 17) find
50. Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. Mart: Improving llm safety with multi-round automatic red-teaming, 2023. (Cited on pages 17)
51. Bhaktipriya Radharapu, Kevin Robinson, Lora Aroyo, and Preethi Lahoti. Aart: Ai-assisted red-teaming with diverse data generation for new llm-powered applications, 2023. (Cited on pages 29) find
52. Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, and Roberta Raileanu. Rainbow teaming: Open-ended generation of diverse adversarial prompts, 2024. (Cited on pages 29) find
53. Nevan Wichers, Carson Denison, and Ahmad Beirami. Gradient-based language model red teaming, 2024. (Cited on pages 17) find
54. Manfred Bierwisch John R. Searle, Ferenc Kiefer. Speech act theory and pragmatics, 1980. (Cited on pages 17) find
55. Kailas Vodrahalli, Tobias Gerstenberg, and James Zou. Uncalibrated models can improve human-ai collaboration, 2022. (Cited on pages 17) find
56. Ko de Ruyter Stephan Ludwig. Decoding social media speak: developing a speech act theory research agenda, 2016. (Cited on pages 18) find
57. Emanuele Arielli. Sharing as speech act, 2018. URL https://philarchive.org/ar chive/ARISAS. ↗
58. Michael Randall Barnes. Who do you speak for? and how?, 2023. URL https://www.rivisteweb.it/doi/10.14649/91354. (Cited on pages 18) — [doi:10.14649/91354](https://doi.org/10.14649/91354) ↗
59. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674, 2023. (Cited on pages 21, 22, 26, 46)
60. Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. Challenges in detoxifying language models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Findin…
61. Alyssa Lees, Vinh Q. Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. A new generation of perspective api: Efficient multilingual character-level transformers, 2022. (Cited on pages 22, 27, 48) find
62. Paul Röttger, Bertie Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet Pierrehumbert. HateCheck: Functional tests for hate speech detection models. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computati…
63. Hannah Kirk, Bertie Vidgen, Paul Rottger, Tristan Thrush, and Scott Hale. Hatemoji: A test suite and adversarially-generated dataset for benchmarking and detecting emoji-based hate. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz, editors, Proceedings of the 2022 Conferen… find
64. Lucas Rosenblatt, Lorena Piedras, and Julia Wilkins. Critical perspectives: A benchmark revealing pitfalls in PerspectiveAPI. In Laura Biester, Dorottya Demszky, Zhijing Jin, Mrinmaya Sachan, Joel Tetreault, Steven Wilson, Lu Xiao, and Jieyu Zhao, editors, Proceedings of the Second Workshop on NLP f… find
65. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017. (Cited on pages 24) find
66. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared… — Constitutional AI: Harmlessness from AI Feedback
67. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. (Cited on pages 24) find
68. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions, 2024. (Cited on pages 26, 27, 30)
69. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. Realchat-1m: A large-scale real-world LLM conversation dataset. In The Twelfth International Conference on Learning Represen… find
70. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. (inthe)wildchat: 570k chatGPT interaction logs in the wild. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum ?id=Bl8u7ZRlbM. (Cited on pages 26, 28) ↗
71. Nathalie A Smuha. Beyond the individual: governing ai’s societal harm. Internet Policy Review, 10(3), 2021. (Cited on pages 26) find
72. Leon Derczynski, Hannah Rose Kirk, Vidhisha Balachandran, Sachin Kumar, Yulia Tsvetkov, M. R. Leiser, and Saif Mohammad. Assessing language model deployment with risk cards, 2023. (Cited on pages 26) find
73. Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N’Mah Yilla-Akbari, Jess Gallegos, Andrew Smart, Emilio Garcia, et al. Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction. In Proceedings of the 2023 AAAI/ACM Conference on AI… find
74. Janna Hastings. Preventing harm from non-conscious bias in medical generative ai. The Lancet Digital Health, 6(1):e2–e3, 2024. (Cited on pages 26) find
75. Barani Maung and Keegan McBride. Unequal Risk, Unequal Reward: How Gen AI disproportionately harms countries. https://www.oii.ox.ac.uk/news-events/unequal-risk-unequal-reward-how-gen-ai-disproportionately-harms-countries/, nov 8 2023. [Online; accessed 2024-04-13]. (Cited on pages 26) ↗
76. Jaemin Cho, Abhay Zala, and Mohit Bansal. DALL-Eval: Probing the reasoning skills and social biases of text-to-image generative transformers. In International Conference of Computer Vision, 2023. (Cited on pages 26) find
77. Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. In Proceedings of the 2023 ACM Conference on Fairn… find
78. Jessica Echterhoff, Yao Liu, Abeer Alessa, Julian McAuley, and Zexue He. Cognitive bias in high-stakes decision-making with llms. arXiv preprint arXiv:2403.00811, 2024. (Cited on pages 26) — [arXiv:2403.00811](https://arxiv.org/abs/2403.00811) find
79. Sonia Livingstone and Mariya Stoilova. The 4cs: Classifying online risk to children, 2021. (Cited on pages 26) find
80. Bertie Vidgen, Emily Burden, and Helen Margetts. Understanding online hate: Vsp regulation and the broader context. Turing Institute, February, https://www.ofcom. org. uk/__data/assets/pdf_file/0022/216490/alan-turing-institute-report-understanding-online-hate. pdf. Accessed, 9, 2021. ↗
81. Heather Frase Mia Hoffmann. Adding structure to ai harm. an introduction to cset’s ai harm framework, 2023. URL https://cset.georgetown.edu/publication/adding -structure-to-ai-harm/. (Cited on pages 26) ↗
82. Jeremy Waldron. Dignity and defamation: The visibility of hate. Harv. L. Rev., 123: 1596, 2009. (Cited on pages 26) find
83. Katharine Gelber and Luke McNamara. Evidencing the harms of hate speech. Social Identities, 22(3):324–341, 2016. doi: 10.1080/13504630.2015.1128810. URL https://doi.org/10.1080/13504630.2015.1128810. — [doi:10.1080/13504630.2015.1128810](https://doi.org/10.1080/13504630.2015.1128810) ↗
84. Jacobo Picardo, Sarah K. McKenzie, Sunny Collings, and Gabrielle Jenkin. Suicide and self-harm content on instagram: A systematic scoping review. PLOS ONE, 15(9):1–16, 09 2020. doi: 10.1371/journal.pone.0238603. URL https://doi.org/10.1371/jo urnal.pone.0238603. — [doi:10.1371/journal.pone.0238603](https://doi.org/10.1371/journal.pone.0238603) find
85. Matt Goerzen, Elizabeth Anne Watkins, and Gabrielle Lim. Entanglements and exploits: Sociotechnical security as an analytic framework. In 9th USENIX Workshop on Free and Open Communications on the Internet (FOCI 19), 2019. (Cited on pages 26) find
86. Julian Hazell. Spear phishing with large language models, 2023. (Cited on pages 26) find
87. Joe Devanny, Huw Dylan, and Elena Grossfeld. Generative ai and intelligence assessment. The RUSI Journal, pages 1–10, 2023. (Cited on pages 26) find
88. Miron Lakomy. Artificial intelligence as a terrorism enabler? understanding the potential impact of chatbots and image generators on online terrorist activities. Studies in Conflict & Terrorism, 0(0):1–21, 2023. doi: 10.1080/1057610X.2023.2259195. URL https://doi.org/10.1080/1057610X.2023.2259195. (… — [doi:10.1080/1057610x.2023.2259195](https://doi.org/10.1080/1057610x.2023.2259195) ↗
89. Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang. Release strategies and the social impacts of language models, 2019. (Cited…
90. Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? arXiv preprint arXiv: 2309.13788, 2023. find
91. Canyu Chen and Kai Shu. Combating misinformation in the age of llms: Opportunities and challenges. arXiv preprint arXiv: 2311.05656, 2023. (Cited on pages 26) find
92. David Thiel, Melissa Stroebel, and Rebecca Portnoff. Generative ml and csam: Implications and mitigations, 2023. (Cited on pages 26) find
93. Julian De Freitas, Ahmet Kaan Uğuralp, Zeliha Oğuz-Uğuralp, and Stefano Puntoni. Chatbots and mental health: insights into the safety of generative ai. Journal of Consumer Psychology, 2022. (Cited on pages 26) find
94. Emilio Ferrara. Genai against humanity: nefarious applications of generative artificial intelligence and large language models. Journal of Computational Social Science, February 2024. ISSN 2432-2725. doi: 10.1007/s42001-024-00250-1. URL http://dx.doi.org/10.1007/s42001-024-00250-1. (Cited on pages… ↗
95. Ashfak Md Shibli, Mir Mehedi A. Pritom, and Maanak Gupta. Abusegpt: Abuse of generative ai chatbots to create smishing campaigns, 2024. (Cited on pages 27) find
96. Maximilian Mozes, Xuanli He, Bennett Kleinberg, and Lewis D. Griffin. Use of llms for illicit purposes: Threats, prevention measures, and vulnerabilities, 2023. (Cited on pages 27) find
97. Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. Felm: Benchmarking factuality evaluation of large language models, 2023. (Cited on pages 27) find
98. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023. (Cited on pages 27)
99. The New Yorker. The terrifying a.i. scam that uses your loved one’s voice, 2024. URL https://www.newyorker.com/science/annals-of-artificial-intelligence/ the-terrifying-ai-scam-that-uses-your-loved-ones-voice. (Cited on pages 27) ↗
100. Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Tauman Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In NIPS, June 2016. URL https://www.microsoft.com/en-us/re search/publication/quantifying-reducing-stereotypes-word-embeddings/. (… ↗
101. Li Lucy and David Bamman. Gender and representation bias in GPT-3 generated stories. In Nader Akoury, Faeze Brahman, Snigdha Chaturvedi, Elizabeth Clark, Mohit Iyyer, and Lara J. Martin, editors, Proceedings of the Third Workshop on Narrative Understanding, pages 48–55, Virtual, June 2021. Associati…
102. Myra Cheng, Esin Durmus, and Dan Jurafsky. Marked personas: Using natural language prompts to measure stereotypes in language models. arXiv preprint arXiv:2305.18189, 2023. (Cited on pages 27)
103. Pranav Gade, Simon Lermen, Charlie Rogers-Smith, and Jeffrey Ladish. Badllama: cheaply removing safety fine-tuning from llama 2-chat 13b, 2024. (Cited on pages 27) find
104. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023. (Cited on pages 27)
105. Ashutosh Kumar, Sagarika Singh, Shiv Vignesh Murty, and Swathy Ragupathy. The ethics of interaction: Mitigating security threats in llms, 2024. (Cited on pages 27) find
106. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024. (Cited on pages 27)
107. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan… — Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
108. Benjamin S. Bucknall and Shiri Dori-Hacohen. Current and near-term ai as a potential existential risk factor. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’22. ACM, July 2022. doi: 10.1145/3514094.3534146. URL http://dx.doi.org/10.1145/3514094.3534146. (Cited on pa… — [doi:10.1145/3514094.3534146](https://doi.org/10.1145/3514094.3534146) ↗
109. Dan Hendrycks, Mantas Mazeika, and Thomas Woodside. An overview of catastrophic ai risks, 2023. find
110. Atoosa Kasirzadeh. Two types of ai existential risk: Decisive and accumulative, 2024. (Cited on pages 27) find
111. Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Katherine Collins, Maryam Molamohammadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstantinos Voudouris, Umang Bh… — [doi:10.1145/3593013.3594033](https://doi.org/10.1145/3593013.3594033) find
112. Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, Lewis Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack Clark, Yoshua Bengio, Paul Christiano, and Al… find
113. Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566, 2024. (Cited on pages 27) find
114. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal, 2024. (Cited on pages 28, 29, 46)
115. Mary Phuong, Matthew Aitchison, Elliot Catt, Sarah Cogan, Alexandre Kaskasoli, Victoria Krakovna, David Lindner, Matthew Rahtz, Yannis Assael, Sarah Hodkinson, Heidi Howard, Tom Lieberum, Ramana Kumar, Maria Abi Raad, Albert Webson, Lewis Ho, Sharon Lin, Sebastian Farquhar, Marcus Hutter, Gregoire D… find
116. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. Towards… — Towards Understanding Sycophancy in Language Models
117. Scott McLean, Gemma JM Read, Jason Thompson, Chris Baber, Neville A Stanton, and Paul M Salmon. The risks associated with artificial general intelligence: A systematic review. Journal of Experimental & Theoretical Artificial Intelligence, 35(5):649–663, 2023. (Cited on pages 27) find
118. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023. (Cited on pages 27)
119. Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi. Auditing large language models: a three-layered approach. AI and Ethics, May 2023. ISSN 2730-5961. doi: 10.1007/s43681-023-00289-2. URL http://dx.doi.org/10.1007/s43681-023-00289-2. (Cited on pages 27, 28) ↗
120. Boming Xia, Qinghua Lu, Liming Zhu, and Zhenchang Xing. Towards ai safety: A taxonomy for ai system evaluation, 2024. (Cited on pages 27) find
121. Traian Rebedea, Razvan Dinu, Makesh Sreedhar, Christopher Parisien, and Jonathan Cohen. Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails, 2023. (Cited on pages 27) find
122. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Das-Sarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shau… find
123. Abeba Birhane, Vinay Prabhu, Sang Han, Vishnu Naresh Boddeti, and Alexandra Sasha Luccioni. Into the laions den: Investigating hate in multimodal datasets, 2023. (Cited on pages 27) find
124. Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O’Keefe, Mark Koren, Théo Ryffel, JB Rubino… find
125. Nijat Rajabli, Francesco Flammini, Roberto Nardone, and Valeria Vittorini. Software verification and validation of safe autonomous cars: A systematic literature review. IEEE Access, 9:4797–4819, 2021. doi: 10.1109/ACCESS.2020.3048047. — [doi:10.1109/access.2020.3048047](https://doi.org/10.1109/access.2020.3048047) find
126. Florian Tambon, Gabriel Laberge, Le An, Amin Nikanjam, Paulina Stevia Nouwou Mindom, Yann Pequignot, Foutse Khomh, Giulio Antoniol, Ettore Merlo, and François Laviolette. How to certify machine learning based safety-critical systems? a systematic literature review. Automated Software Engineering, 29… — [doi:10.1007/s10515-022-00337-x](https://doi.org/10.1007/s10515-022-00337-x) find
127. Thomas G. Dietterich. Steps toward robust artificial intelligence. AI Magazine, 38(3): 3–24, Oct. 2017. doi: 10.1609/aimag.v38i3.2756. URL https://ojs.aaai.org/aim agazine/index.php/aimagazine/article/view/2756. (Cited on pages 27) — [doi:10.1609/aimag.v38i3.2756](https://doi.org/10.1609/aimag.v38i3.2756) ↗
128. Mubashara Akhtar, Omar Benjelloun, Costanza Conforti, Joan Giner-Miguelez, Nitisha Jain, Michael Kuchnik, Quentin Lhoest, Pierre Marcenac, Manil Maskey, Peter Mattson, Luis Oala, Pierre Ruyssen, Rajat Shinde, Elena Simperl, Goeffry Thomas, Slava Tykhonov, Joaquin Vanschoren, Steffen Vogler, and Caro… find
129. Luis Oala, Manil Maskey, Lilith Bat-Leah, Alicia Parrish, Nezihe Merve Gürel, Tzu-Sheng Kuo, Yang Liu, Rotem Dror, Danilo Brajovic, Xiaozhe Yao, Max Bartolo, William A Gaviria Rojas, Ryan Hileman, Rainier Aliment, Michael W. Mahoney, Meg Risdal, Matthew Lease, Wojciech Samek, Debojyoti Dutta, Curtis… find
130. Lora Aroyo, Alex S. Taylor, Mark Diaz, Christopher M. Homan, Alicia Parrish, Greg Serapio-Garcia, Vinodkumar Prabhakaran, and Ding Wang. Dices dataset: Diversity in conversational ai evaluation for safety, 2023. (Cited on pages 27) find
131. Context Fund Policy Working Group. NTIA Open Weights Response: Towards A Secure Open Society Powered By Personal AI, 2024. URL https://www.context.fund/pol icy/ntia_open_weights_response.html. (Cited on pages 28) ↗
132. Robert M Bond, Christopher J Fariss, Jason J Jones, Adam DI Kramer, Cameron Marlow, Jaime E Settle, and James H Fowler. A 61-million-person experiment in social influence and political mobilization. Nature, 489(7415):295–298, 2012. (Cited on pages 28) find
133. Maurice Jakesch, Megan French, Xiao Ma, Jeffrey T. Hancock, and Mor Naaman. Ai-mediated communication: How the perception that profile text was written by ai affects trustworthiness. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI ’19, page 1–13, New York, NY, US… — [doi:10.1145/3290605.3300469](https://doi.org/10.1145/3290605.3300469) find
134. Mark Ledwich and Anna Zaitsev. Algorithmic extremism: Examining youtube’s rabbit hole of radicalization, 2019. find
135. Robert Gorwa, Reuben Binns, and Christian Katzenbach. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, 7(1):2053951719897945, 2020. doi: 10.1177/2053951719897945. URL https://doi.org/10.1177/2053951719897945. — [doi:10.1177/2053951719897945](https://doi.org/10.1177/2053951719897945) ↗
136. J Hohenstein, D DiFranzo, RF Kizilcec, Z Aghajari, H Mieczkowski, K Levy, M Naaman, J Hancock, and M Jung. Artificial intelligence in communication impacts language and social relationships. arxiv. arXiv preprint arXiv:2102.05756, 2021. (Cited on pages 28) — [arXiv:2102.05756](https://arxiv.org/abs/2102.05756) find
137. Homa Hosseinmardi, Amir Ghasemian, Miguel Rivera-Lanas, Manoel Horta Ribeiro, Robert West, and Duncan J. Watts. Causally estimating the effect of youtube’s recommender system using counterfactual bots. Proceedings of the National Academy of Sciences, 121(8):e2313377121, 2024. doi: 10.1073/pnas.23133… — [doi:10.1073/pnas.2313377121](https://doi.org/10.1073/pnas.2313377121) find
138. Malak Abdullah, Alia Madain, and Yaser Jararweh. Chatgpt: Fundamentals, applications and social impacts. In 2022 Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS), pages 1–8, 2022. doi: 10.1109/SNAMS58071.2022.10062688. (Cited on pages 28) — [doi:10.1109/snams58071.2022.10062688](https://doi.org/10.1109/snams58071.2022.10062688) find
139. Abigail Z. Jacobs and Hanna Wallach. Measurement and fairness. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 375–385, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383097. doi: 10.1145/3442188.3445901. URL https:/… find
140. Sayash Kapoor, Rishi Bommasani, Kevin Klyman, Shayne Longpre, Ashwin Ramaswami, Peter Cihon, Aspen Hopkins, Kevin Bankston, Stella Biderman, Miranda Bogen, Rumman Chowdhury, Alex Engler, Peter Henderson, Yacine Jernite, Seth Lazar, Stefano Maffulli, Alondra Nelson, Joelle Pineau, Aviya Skowron, Dawn… find
141. Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, and Jiawei Han. The shifted and the overlooked: A task-oriented investigation of user-gpt interactions, 2023. (Cited on pages 28) find
142. Luis Oala, Jana Fehr, Luca Gilli, Pradeep Balachandran, Alixandro Werneck Leite, Saul Calderon-Ramirez, Danny Xie Li, Gabriel Nobis, Erick Alejandro Muñoz Alvarado, Giovanna Jaramillo-Gutierrez, Christian Matek, Arun Shroff, Ferath Kherif, Bruno Sanguinetti, and Thomas Wiegand. Ml4h auditing: From p… find
143. Gregory Falco, Ben Shneiderman, Julia Badger, Ryan Carrier, Anton Dahbura, David Danks, Martin Eling, Alwyn Goodloe, Jerry Gupta, Christopher Hart, et al. Governing ai safety through independent audits. Nature Machine Intelligence, 3(7):566–571, 2021. find
144. Heidy Khlaaf. Toward comprehensive risk assessments and assurance of ai-based systems. Trail of Bits, 2023. find
145. Lee Sharkey, Clíodhna Ní Ghuidhir, Dan Braun, Jérémy Scheurer, Mikita Balesni, Lucius Bushnaq, Charlotte Stix, and Marius Hobbhahn. A causal framework for ai regulation and auditing, 2024. (Cited on pages 28) find
146. Khoa Lam, Benjamin Lange, Borhane Blili-Hamelin, Jovana Davidovic, Shea Brown, and Ali Hasan. A Framework for Assurance Audits of Algorithmic Systems. arXiv preprint arXiv:2401.14908, Forthcoming. URL http://arxiv.org/abs/2401.14908. arXiv:2401.14908 [cs]. (Cited on pages 28) — [arXiv:2401.14908](https://arxiv.org/abs/2401.14908) ↗
147. Andrew D Selbst, Danah Boyd, Sorelle A Friedler, Suresh Venkatasubramanian, and Janet Vertesi. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency, pages 59–68, 2019. (Cited on pages 28) find
148. Jacob Metcalf, Emanuel Moss, Ranjit Singh, Emnet Tafese, and Elizabeth Anne Watkins. A relationship and not a thing: A relational approach to algorithmic accountability and assessment documentation. arXiv preprint arXiv:2203.01455, 2022. (Cited on pages 28) — [arXiv:2203.01455](https://arxiv.org/abs/2203.01455) find
149. Jakob Mökander and Luciano Floridi. Ethics-based auditing to develop trustworthy ai. Minds and Machines, 31(2):323–327, 2021. (Cited on pages 28) find
150. Luis Oala, Andrew G Murchison, Pradeep Balachandran, Shruti Choudhary, Jana Fehr, Alixandro Werneck Leite, Peter G Goldschmidt, Christian Johner, Elora DM Schörverth, Rose Nakasi, et al. Machine learning for health: algorithm auditing & quality control. Journal of medical systems, 45:1–8, 2021. (Cit… find
151. Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities a… find
152. Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji,… — TrustLLM: Trustworthiness in Large Language Models
153. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. Decodingtrust: A comprehensive assessment of trus… — DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
154. Rohan Anil et al. Palm 2 technical report, 2023. (Cited on pages 29)
155. Simon Ott, Adriano Barbosa-Silva, Kathrin Blagec, Jan Brauner, and Matthias Samwald. Mapping global dynamics of benchmark creation and saturation in artificial intelligence. Nature Communications, 13(1), November 2022. ISSN 2041-1723. doi: 10.1038/s414 67-022-34591-0. URL http://dx.doi.org/10.1038/s…
156. Joseph R Biden. Executive order on the safe, secure, and trustworthy development and use of artificial intelligence, 2023. (Cited on pages 29) find
157. Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety-aligned llms with simple adaptive attacks, 2024. (Cited on pages 29) find
158. Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Liu Kost, Christopher Carnahan, and Jordan Boyd-Graber. Ignore this title and hackaprompt: Exposing systemic vulnerabilities of llms through a global scale prompt hacking competit… find
159. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms, 2024. (Cited on pages 29)
160. Victor Storchan, Ravin Kumar, Rumman Chowdhury, Seraphina Goldfarb-Tarrant, and Sven Cattell. Generative ai red teaming challenge: transparency report, 2024. URL https://drive.google.com/file/d/1JqpbIP6DNomkb32umLoiEPombK2-0Rc-/view. (Cited on pages 29) ↗
161. Peter Mattson, Christine Cheng, Cody Coleman, Greg Diamos, Paulius Micikevicius, David Patterson, Hanlin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debojyoti Dutta, Udit Gupta, Kim Hazelwood, Andrew Hock, Xinyuan Huang, Atsushi Ike, Bill Jia, Daniel Kang, David Kanter… find
162. Avrim Blum and Moritz Hardt. The ladder: A reliable leaderboard for machine learning competitions. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 1006–1014, Lille, France, 07–… find
163. Harm de Vries, Dzmitry Bahdanau, and Christopher Manning. Towards ecologically valid research on language user interfaces, 2020. (Cited on pages 29) find
164. Samuel R. Bowman and George E. Dahl. What will it take to fix benchmarking in natural language understanding?, 2021. (Cited on pages 29) find
165. Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. Dynabench:… find
166. Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, and Jingbo Shang. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation, 2023. (Cited on pages 30)
167. Rishabh Bhardwaj and Soujanya Poria. Red-teaming large language models using chain of utterances for safety-alignment, 2023. (Cited on pages 30) find
168. Megan Kinniment, Brian Goodrich, Max Hasin, Ryan Bloom, Haoxing Du, Lucas Jun Koba Sato, Daniel Ziegler, Timothee Chauvin, Thomas Broadley, Tao R. Lin, Ted Suzman, Francisco Carvalho, Michael Chen, Niels Warncke, Bart Bussmann, Axel Højmark, Chris MacLeod, and Elizabeth Barnes. Metr example task sui… find
169. ActiveFence. Activefence safety api, 2024. URL https://www.activefence.com/ac tive-score/. (Cited on pages 46) ↗
170. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment, 2024. (Cited on pages 47)
171. Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Chi Zhang, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of llm via a human-preference dataset, 2023. (Cited on pages 48)
172. Unitary AI. Unitary ai, detoxify, 2021. URL https://github.com/unitaryai/detox ify. (Cited on pages 48) ↗
173. SalesForce. Auditnlg: Auditing generative ai language modeling for trustworthiness, 2023. URL https://github.com/salesforce/AuditNLG. (Cited on pages 48) ↗
174. Google Vertex AI. Configure safety settings for the palm api, 2024. URL https://cloud.google.com/vertex-ai/generative-ai/docs/configure-safety-att ributes-palm. (Cited on pages 49) ↗
175. Hive AI. Content moderation ai, 2024. URL https://thehive.ai/. (Cited on pages 49) ↗
176. Todor Markov, Chong Zhang, Sandhini Agarwal, Tyna Eloundou, Teddy Lee, Steven Adler, Angela Jiang, and Lilian Weng. A holistic approach to undesired content detection in the real world, 2023. (Cited on pages 49)
177. Microsoft Azure AI. Content safety filters, 2024. URL https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety. (Cited on pages 49) ↗
178. Writer AI. Toxic check, 2024. URL https://dev.writer.com/docs/toxic-check. (Cited on pages 49) ↗
179. Erica Chenoweth, Richard English, Andreas Gofas, and Stathis N. Kalyvas. The oxford handbook of terrorism, 2019. URL https://books.google.co.uk/books?hl=en&l r=&id=lu-MDwAAQBAJ&. (Cited on pages 50) ↗
180. John Horgan Donald Holbrook. Terrorism and ideology: Cracking the nut, 2019. URL https://www.jstor.org/stable/26853737?seq=6. (Cited on pages 50) ↗

Updated 1 November 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/vidgen-2024-introducing-v0-5-ai-safety/