Elamur Research⌕
☰
home / papers / 2024 / OR-Bench: An Over-Refusal Benchmark for Large Language Models

OR-Bench: An Over-Refusal Benchmark for Large Language Models

Paper · 2024 · Justin Cui, Wei-Lin Chiang, Ion Stoica et al.

Summary

Safety-tuned LLMs often refuse harmless requests that merely look risky. Before this work, the problem had no large-scale way to be measured. The only dedicated test set, XSTest, has 250 hand-written prompts, and recent strong models now answer almost all of them. The authors build an automated pipeline to fill this gap. First, a less restricted model (Mixtral 8*7B) produces harmful seed prompts for each refusal category. Next, an LLM rewrites each seed into a benign but borderline version. Finally, an ensemble of LLM moderators removes any rewrites that are still toxic. The result is OR-Bench, which has three parts: a broad over-refusal set, a hard subset that even frontier models struggle with, and a toxic control set that penalizes models that answer everything. The authors test 32 LLMs and find that safety and over-refusal rise together: models that block more toxic requests also reject more benign ones. The benchmark gives the field a renewable, model-agnostic testbed for the trade-off between safety and helpfulness. The paper was published at ICML 2025.

Key points

  • Resources: OR-Bench-80K has 80,000 seemingly risky but safe prompts across 10 categories, such as violence, privacy, hate and sexual content. OR-Bench-Hard-1K has about 1,000 difficult prompts, and OR-Bench-Toxic has 600 harmful prompts. The data is on Hugging Face (bench-llms) and the code is on GitHub (justincui03/or-bench).
  • Evaluation scope: The authors test 32 open-source and black-box LLMs from 8 families. On the hard and toxic sets, the Spearman rank correlation between toxic-prompt rejection and over-refusal is 0.89.
  • Model findings:
    • Claude models are the safest but refuse the most benign prompts, while Mistral models accept most requests.
    • Newer GPT-3.5-turbo versions refuse less but are also less safe.
    • Larger models do not necessarily strike a better balance.
  • Defenses and system prompts:
    • Jailbreak defenses such as ICL and SmoothLLM raise over-refusal; ICL rejects the most of both toxic and benign prompts.
    • A system prompt asking for safe, helpful answers made GPT-3.5-turbo-0125 reject about 35% more toxic prompts and about 55% more benign ones. For Mistral-large-latest, the figures were about 20% and 10%.
  • Robustness: Sampling temperature barely changes the results. For example, Claude-3-Haiku's rejection rate on the hard subset moves only from 96.2 to 95.5 across temperatures 0.0–1.0.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksSocial Sciences · 744 worksSafety Research · 221 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 1,343 worksEvaluation & benchmarks · 685 worksBias, fairness & safety · 209 works

Cite

Cite as Justin Cui, Wei-Lin Chiang, Ion Stoica and Cho-Jui Hsieh. 2024. OR-Bench: An Over-Refusal Benchmark for Large Language Models. arXiv:2405.20947.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2405.20947 · arXiv 2405.20947

@misc{cui2024or,
  title = {{OR-Bench: An Over-Refusal Benchmark for Large Language Models}},
  author = {Justin Cui, Wei-Lin Chiang, Ion Stoica and Cho-Jui Hsieh},
  year = {2024},
  doi = {10.48550/arxiv.2405.20947},
  eprint = {2405.20947},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2405.20947}
}

References

83 references, 42 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. GPT-4 AI is Great, But at a Hefty Price Tag ;) — community.openai.com. https://community.openai.com, a. Gpt-4-0125-preview INCREDIBLY slower than 3.5 turbo. https://community.openai.com/, b. Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., and Wong, E. Jailbreaking black box large lan… ↗
  2. Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https://lmsys.org/blog/ 2023-03-30-vicuna/. ↗
  3. Abdin, M., Jacobs, S. A., Awan, A. A., Aneja, J., Awadallah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H., et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024.
  4. Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024.
  5. An, B., Zhu, S., Zhang, R., Panaitescu-Liess, M.-A., Xu, Y., and Huang, F. Automatic pseudo-harmful prompt generation for evaluating false refusals in large language models. arXiv preprint arXiv:2409.00598, 2024. — [arXiv:2409.00598](https://arxiv.org/abs/2409.00598) find
  6. Claude. Claude 2.1 Refuses to kill a Python process — Hacker News — news.ycombinator.com. https://news.ycombinator.com/item?id=38371115, 2023. [Accessed 08-05-2024]. ↗
  7. Anthropic. Introducing the next generation of Claude — anthropic.com. https://www.anthropic.com/ news/claude-3-family, 2024a. [Accessed 07-05-2024]. ↗
  8. Claude3. Introducing the next generation of claude \\ anthropic. https://www.anthropic.com/news/ claude-3-family, 2024. ↗
  9. Anthropic. Claude 3 addendum.pdf. https://www-cdn.anthropic.com/Model_Card_ Claude_3_Addendum.pdf, 2024b. ↗
  10. Dai, J., Pan, X., Sun, R., Ji, J., Xu, X., Liu, M., Wang, Y., and Yang, Y. Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXiv:2310.12773, 2023.
  11. Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. Foundational challenges in assuring alignment and safety of large language models. arXiv preprint arXiv:2404.09932, 2024. find
  12. Davidson, T., Bhattacharya, D., and Weber, I. Racial bias in hate speech and abusive language detection datasets. arXiv preprint arXiv:1905.12516, 2019.
  13. Dixon, L., Li, J., Sorensen, J., Thain, N., and Vasserman, L. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 67–73, 2018. find
  14. Jiang, L., Rao, K., Han, S., Ettinger, A., Brahman, F., Kumar, S., Mireshghallah, N., Lu, X., Sap, M., Choi, Y., et al. Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Advances in Neural Information Processing Systems, 37:47094–47165, 2024b.
  15. Dong, Q., Li, L., Dai, D., Zheng, C., Wu, Z., Chang, B., Sun, X., Xu, J., and Sui, Z. A survey on in-context learning. arXiv preprint arXiv:2301.00234, 2022.
  16. Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
  17. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. find
  18. Kim, H., Yu, Y., Jiang, L., Lu, X., Khashabi, D., Kim, G., Choi, Y., and Sap, M. Prosocialdialog: A prosocial backbone for conversational agents. arXiv preprint arXiv:2205.12688, 2022.
  19. Feuer, B., Goldblum, M., Datta, T., Nambiar, S., Besaleli, R., Dooley, S., Cembalest, M., and Dickerson, J. P. Style over substance: Failure modes of llm judges in alignment benchmarking. arXiv preprint arXiv:2409.15268, 2024. — [arXiv:2409.15268](https://arxiv.org/abs/2409.15268) find
  20. Koike, R., Kaneko, M., and Okazaki, N. Outfox: Llm-generated essay detection through in-context learning with adversarially generated examples. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 21258–21266, 2024. find
  21. Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al. Red teaming language models to reduce harms: methods, scaling behaviors, and lessons learned. arxiv, 2022. less. Refusal in LLMs is mediated by a single direction — LessWro…
  22. Ge, S., Zhou, C., Hou, R., Khabsa, M., Wang, Y.-C., Wang, Q., Han, J., and Mao, Y. Mart: Improving llm safety with multi-round automatic red-teaming. arXiv preprint arXiv:2311.07689, 2023.
  23. Levy, S., Allaway, E., Subbiah, M., Chilton, L., Patton, D., McKeown, K., and Wang, W. Y. Safetext: A benchmark for exploring physical safety in language models. arXiv preprint arXiv:2210.10045, 2022. find
  24. Han, S., Rao, K., Ettinger, A., Jiang, L., Lin, B. Y., Lambert, N., Choi, Y., and Dziri, N. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. arXiv preprint arXiv:2406.18495, 2024.
  25. Li, L., Dong, B., Wang, R., Hu, X., Zuo, W., Lin, D., Qiao, Y., and Shao, J. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models. arXiv preprint arXiv:2402.05044, 2024a.
  26. Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., and Kamar, E. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. arXiv preprint arXiv:2203.09509, 2022.
  27. Li, X., Wang, R., Cheng, M., Zhou, T., and Hsieh, C.-J. Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers. arXiv preprint arXiv:2402.16914, 2024b. — [arXiv:2402.16914](https://arxiv.org/abs/2402.16914) find
  28. Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674, 2023.
  29. Lin, Z., Wang, Z., Tong, Y., Wang, Y., Guo, Y., Wang, Y., and Shang, J. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation. arXiv preprint arXiv:2310.17389, 2023.
  30. Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
  31. Ji, J., Liu, M., Dai, J., Pan, X., Zhang, C., Bian, C., Chen, B., Sun, R., Wang, Y., and Yang, Y. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36, 2024.
  32. Llama3. Introducing meta llama 3: The most capable openly available llm to date. https://ai.meta.com/blog/meta-llama-3/, 2024. ↗
  33. Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024a. find
  34. Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020.
  35. McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., and Halgamuge, M. N. Inadequacies of large language model benchmarks in the era of generative artificial intelligence. arXiv preprint arXiv:2402.09880, 2024. — [arXiv:2402.09880](https://arxiv.org/abs/2402.09880) find
  36. Röttger, P., Kirk, H. R., Vidgen, B., Attanasio, G., Bianchi, F., and Hovy, D. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint arXiv:2308.01263, 2023. mistral. mistralai/Mistral-7B-Instruct-v0.3 · Hugging Face — huggingface.co. https://hug…
  37. Samvelyan, M., Raparthy, S. C., Lupu, A., Hambro, E., Markosyan, A. H., Bhatt, M., Mao, Y., Jiang, M., Parker-Holder, J., Foerster, J., et al. Rainbow teaming: Open-ended generation of diverse adversarial prompts. arXiv preprint arXiv:2402.16822, 2024. — [arXiv:2402.16822](https://arxiv.org/abs/2402.16822) find
  38. Mistral. Mistral AI — Frontier AI in your hands — mistral.ai. https://mistral.ai/, 2024. [Accessed 07-05-2024]. ↗
  39. Sap, M., Card, D., Gabriel, S., Choi, Y., and Smith, N. A. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 1668–1678, 2019.
  40. OpenAI. Chatgpt. https://www.openai.com, 2023. Accessed: ¡date-of-access¿. ↗
  41. Stiennon, N., Ouyang, L., Wu, J., Ziegler, D., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P. F. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33: 3008–3021, 2020. find
  42. OpenAI. Introducing the model spec — openai. 2024. find
  43. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022.
  44. Sun, L., Huang, Y., Wang, H., Wu, S., Zhang, Q., Gao, C., Huang, Y., Lyu, W., Zhang, Y., Li, X., et al. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.05561, 2024.
  45. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318, 2002.
  46. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/ stanford_alpaca, 2023a. ↗
  47. Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023.
  48. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm.stanford. edu/2023/03/13/alpaca. html, 3(6):7, 2023b. ↗
  49. Qiu, H., Zhang, S., Li, A., He, H., and Lan, Z. Latent jailbreak: A benchmark for evaluating text safety and output robustness of large language models. arXiv preprint arXiv:2307.08487, 2023. — [arXiv:2307.08487](https://arxiv.org/abs/2307.08487) find
  50. Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024. find
  51. Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
  52. Tedeschi, S., Friedrich, F., Schramowski, P., Kersting, K., Navigli, R., Nguyen, H., and Li, B. Alert: A comprehensive benchmark for assessing large language models’ safety through red teaming. arXiv preprint arXiv:2404.08676, 2024. — [arXiv:2404.08676](https://arxiv.org/abs/2404.08676) find
  53. Reid, M., Savinov, N., Teplyashin, D., Lepikhin, D., Lillicrap, T., Alayrac, J.-b., Soricut, R., Lazaridou, A., Firat, O., Schrittwieser, J., et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024.
  54. Thakur, A. S., Choudhary, K., Ramayapally, V. S., Vaidyanathan, S., and Hupkes, D. Judging the judges: Evaluating alignment and vulnerabilities in llms-as-judges. arXiv preprint arXiv:2406.12624, 2024. find
  55. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a.
  56. Robey, A., Wong, E., Hassani, H., and Pappas, G. J. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684, 2023.
  57. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023b. find
  58. Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., and Dinan, E. Recipes for safety in open-domain chatbots. arXiv preprint arXiv:2010.07079, 2020. find
  59. Xu, L., Zhao, K., Zhu, L., and Xue, H. Sc-safety: A multi-round open-ended question adversarial safety benchmark for large language models in chinese. arXiv preprint arXiv:2310.05818, 2023. — [arXiv:2310.05818](https://arxiv.org/abs/2310.05818) find
  60. Tuan, Y.-L., Chen, X., Smith, E. M., Martin, L., Batra, S., Celikyilmaz, A., Wang, W. Y., and Bikel, D. M. Towards safety and helpfulness balanced responses via controllable large language models. arXiv preprint arXiv:2404.01295, 2024. — [arXiv:2404.01295](https://arxiv.org/abs/2404.01295) find
  61. Xu, Z., Liu, Y., Deng, G., Li, Y., and Picek, S. Llm jailbreak attack versus defense techniques–a comprehensive study. arXiv preprint arXiv:2402.13457, 2024. — [arXiv:2402.13457](https://arxiv.org/abs/2402.13457) find
  62. Wang, B., Xu, C., Wang, S., Gan, Z., Cheng, Y., Gao, J., Awadallah, A. H., and Li, B. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2111.02840, 2021. find
  63. Yu, Y., Zhuang, Y., Zhang, J., Meng, Y., Ratner, A. J., Krishna, R., Shen, J., and Zhang, C. Large language model as attributed training data generator: A tale of diversity and bias. Advances in Neural Information Processing Systems, 36, 2024.
  64. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022.
  65. Yuan, T., He, Z., Dong, L., Wang, Y., Zhao, R., Xia, T., Xu, L., Zhou, B., Li, F., Zhang, Z., et al. R-judge: Benchmarking safety risk awareness for llm agents. arXiv preprint arXiv:2401.10019, 2024.
  66. Wang, Y., Shi, Z., Bai, A., and Hsieh, C.-J. Defending llms against jailbreaking attacks via backtranslation. arXiv preprint arXiv:2402.16459, 2024. find
  67. Wei, A., Haghtalab, N., and Steinhardt, J. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024.
  68. Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., Radharapu, B., et al. Shieldgemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772, 2024a. — [arXiv:2407.21772](https://arxiv.org/abs/2407.21772) find
  69. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682, 2022a.
  70. Zeng, Y., Lin, H., Zhang, J., Yang, D., Jia, R., and Shi, W. How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms. arXiv preprint arXiv:2401.06373, 2024b.
  71. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022b.
  72. Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wu, F., et al. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792, 2023.
  73. Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. find
  74. Wei, Z., Wang, Y., and Wang, Y. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387, 2023.
  75. Zheng, C., Yin, F., Zhou, H., Meng, F., Zhou, J., Chang, K.-W., Huang, M., and Peng, N. On prompt-driven safeguarding for large language models. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models. find
  76. Xie, T., Qi, X., Zeng, Y., Huang, Y., Sehwag, U. M., Huang, K., He, L., Wei, B., Li, D., Sheng, Y., et al. Sorry-bench: Systematically evaluating large language model safety refusal behaviors. arXiv preprint arXiv:2406.14598, 2024a. find
  77. Zheng, L., Chiang, W.-L., Sheng, Y., Li, T., Zhuang, S., Wu, Z., Zhuang, Y., Li, Z., Lin, Z., Xing, E., et al. Lmsys-chat-1m: A large-scale real-world llm conversation dataset. arXiv preprint arXiv:2309.11998, 2023.
  78. Xie, X., Song, J., Zhou, Z., Huang, Y., Song, D., and Ma, L. Online safety analysis for llms: a benchmark, an assessment, and a path forward. arXiv preprint arXiv:2404.08517, 2024b. — [arXiv:2404.08517](https://arxiv.org/abs/2404.08517) find
  79. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024. find
  80. Xie, Y., Yi, J., Shao, J., Curl, J., Lyu, L., Chen, Q., Xie, X., and Wu, F. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12): 1486–1496, 2023. find
  81. Zhou, X. Challenges in automated debiasing for toxic language detection. University of Washington, 2020.
  82. Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., Yang, L., Ye, W., Gong, N. Z., Zhang, Y., et al. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv preprint arXiv:2306.04528, 2023.
  83. Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cui-2024-bench-over-refusal-benchmark-large/