Elamur Research⌕
☰
home / papers / 2024 / Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models

Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models

Paper · 2024 · Yingshui Tan, Boren Zheng, Baihui Zheng et al.

Summary

Models deployed in China must handle questions about local law, policy and ethics correctly. Existing safety benchmarks mostly use red-teaming or case-based prompts that mix several risks at once, which makes it hard to tell which areas of safety knowledge a model lacks. The authors introduce Chinese SafetyQA, a short-form factuality benchmark modelled on SimpleQA and Chinese SimpleQA. It tests what a model knows about safety rather than whether it can be pushed into unsafe output. The 2,000 harmless questions come from search engines, official Chinese websites and expert-written material. GPT-4o expanded these sources into QA and MCQ items. The items then went through LLM checks, RAG-based answer checks, safety-rule screening, difficulty filtering and dual human annotation. Ideological and political topics were left out on purpose. Tests on more than 30 LLMs show that most have weak safety factuality and tend to be overconfident. Retrieval helps, but self-reflection does not. The benchmark gives developers a cheap way to find gaps in safety knowledge relevant to compliance.

Key points

  • Size and structure: The benchmark has 2,000 items, each available as a short-form QA and as an MCQ (4,000 entries in total). They are organised into a 7–27–103 hierarchy of primary, secondary and fine-grained topics. The seven primary topics are EM, IH, PD, RM, IRC, PMH and STK.
  • Difficulty filtering: Four models were run on each item: o1-preview, Qwen-max, Claude-3.5-Sonnet and Gemini-1.5-pro. Any item that all four answered correctly was removed. Gold answers must be unique, stable over time and answerable as of the end of 2023.
  • Overall results: o1-preview performed best, scoring above 60 in every category. GPT-4o-mini performed worst. Chinese models such as the Qwen series and Doubao did better on legal compliance (IRC) and rumors (RM) but were weak on safety theory (STK).
  • RAG findings: Passive RAG beat active, model-triggered RAG. Smaller models gained the most, for example Qwen2.5-3B compared with Qwen2.5-72B. The gains were smaller than those seen on general-knowledge benchmarks.
  • Other findings: Models showed a "tip-of-the-tongue" effect and poorly calibrated confidence. Self-reflection sometimes added new errors instead of fixing old ones.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Alibaba GroupHangzhou · China168 works · 15,386 citations🔬China Academy of Information and Communications TechnologyBeijing · China2 works · 5 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEvaluation & benchmarks · 1,220 worksBias, fairness & safety · 517 works

Cite

Cite as Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiangyong Zhu, Bo Zheng and Kaifu Zhang. 2024. Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models. arXiv:2412.15265.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2412.15265 · arXiv 2412.15265

@misc{tan2024chinese,
  title = {{Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models}},
  author = {Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiangyong Zhu, Bo Zheng and Kaifu Zhang},
  year = {2024},
  doi = {10.48550/arxiv.2412.15265},
  eprint = {2412.15265},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2412.15265}
}

References

45 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. L., Tan, L., Jenkins, L., Martin, L., Madaan, L., Malo, L., Blecher, L., Landzaat, L., de Oliveira, L., Muzzi, M., Pasupuleti, M., Singh, M., Paluri, M., Kardas, M., Oldham, M., Rita, M., Pavlova, M., Kambadur, M., Lewis, M., Si, M., Singh, M. K., Hassan, M., Goyal, N., Torabi, N., Bashlykov, N., Bo… — Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
  2. Baichuan. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305, 2023. URL https://arxiv.org/abs/2309.10305. ↗
  3. Brown, R. and McNeill, D. The “tip of the tongue” phenomenon. Journal of verbal learning and verbal behavior, 5(4):325–337, 1966. find
  4. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901, 2020.
  5. DeepSeek-AI. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954, 2024a. URL https://github.com/deepseek-ai/DeepSeek-LLM. ↗
  6. DeepSeek-AI. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model, 2024b. find
  7. Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., and Tang, J. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 320–335, 2022.
  8. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Roziere, B., Biron, B., Tang… — The Llama 3 Herd of Models
  9. Yang, S., Tam, W. L., Zhao, W., Liu, X., Xia, X., Zhang, X., Gu, X., Lv, X., Liu, X., Liu, X., Yang, X., Song, X., Zhang, X., An, Y., Xu, Y., Niu, Y., Yang, Y., Li, Y., Bai, Y., Dong, Y., Qi, Z., Wang, Z., Yang, Z., Du, Z., Hou, Z., and Wang, Z. Chatglm: A family of large language models from glm-13… find
  10. He, Y., Li, S., Liu, J., Tan, Y., Wang, W., Huang, H., Bu, X., Guo, H., Hu, C., Zheng, B., Lin, Z., Liu, X., Sun, D., Lin, S., Zheng, Z., Zhu, X., Su, W., and Zheng, B. Chinese simpleqa: A chinese factuality evaluation for large language models, 2024a. URL https://arxiv.org/abs/2411.07140. ↗
  11. He, Y., Li, S., Liu, J., Tan, Y., Wang, W., Huang, H., Bu, X., Guo, H., Hu, C., Zheng, B., et al. Chinese simpleqa: A chinese factuality evaluation for large language models. arXiv preprint arXiv:2411.07140, 2024b.
  12. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
  13. Huang, Y., Bai, Y., Zhu, Z., Zhang, J., Zhang, J., Su, T., Liu, J., Lv, C., Zhang, Y., Lei, J., Fu, Y., Sun, M., and He, J. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. arXiv preprint arXiv:2305.08322, 2023.
  14. Ji, J., Liu, M., Dai, J., Pan, X., Zhang, C., Bian, C., Chen, B., Sun, R., Wang, Y., and Yang, Y. Beavertails: Towards improved safety alignment of llm via a human-preference dataset. Advances in Neural Information Processing Systems, 36, 2024.
  15. Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., and Fung, P. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38, 2023a.
  16. Ji, Z., Yu, T., Xu, Y., Lee, N., Ishii, E., and Fung, P. Towards mitigating llm hallucination via self reflection. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 1827–1843, 2023b.
  17. Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., and Li, Q. A survey on rag meeting llms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6491–6501, 2024.
  18. Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023a. find
  19. Farquhar, S., Varma, V., Kenton, Z., Gasteiger, J., Mikulik, V., and Shah, R. Challenges with unsupervised llm knowledge discovery. arXiv preprint arXiv:2312.10029, 2023. — [arXiv:2312.10029](https://arxiv.org/abs/2312.10029) find
  20. Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., and Neubig, G. Active retrieval augmented generation. arXiv preprint arXiv:2305.06983, 2023b.
  21. GLM, T., Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Rojas, D., Feng, G., Zhao, H., Lai, H., Yu, H., Wang, H., Sun, J., Zhang, J., Cheng, J., Gui, J., Tang, J., Zhang, J., Li, J., Zhao, L., Wu, L., Zhong, L., Liu, M., Huang, M., Zhang, P., Zheng, Q., Lu, R., Duan, S., Zhang, S., Cao, S., Jiao, J… — [arXiv:2406.18841](https://arxiv.org/abs/2406.18841) find
  22. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459–9474, 2020.
  23. Tihanyi, N., Ferrag, M. A., Jain, R., Bisztray, T., and Debbah, M. Cybermetric: A benchmark dataset based on retrieval-augmented generation for evaluating llms in cybersecurity knowledge, 2024. URL https://arxiv.org/abs/2402.07688. — [arXiv:2402.07688](https://arxiv.org/abs/2402.07688) ↗
  24. Li, H., Zhang, Y., Koto, F., Yang, Y., Zhao, H., Gong, Y., Duan, N., and Baldwin, T. Cmmlu: Measuring massive multitask language understanding in chinese, 2023.
  25. Wang, C., Liu, X., Yue, Y., Tang, X., Zhang, T., Jiayang, C., Yao, Y., Gao, W., Hu, X., Qi, Z., et al. Survey on factuality in large language models: Knowledge, retrieval and domain-specificity. arXiv preprint arXiv:2310.07521, 2023a.
  26. Li, L., Dong, B., Wang, R., Hu, X., Zuo, W., Lin, D., Qiao, Y., and Shao, J. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models, 2024. URL https://arxiv.org/abs/2402. 05044. ↗
  27. Lin, Z., Wang, Z., Tong, Y., Wang, Y., Guo, Y., Wang, Y., and Shang, J. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation, 2023. URL https://arxiv.org/abs/2310.17389. ↗
  28. Wang, Y., Li, H., Han, X., Nakov, P., and Baldwin, T. Do-not-answer: A dataset for evaluating safeguards in llms, 2023b. URL https://arxiv.org/abs/ 2308.13387. ↗
  29. Wang, Y., Wang, M., Iqbal, H., Georgiev, G., Geng, J., and Nakov, P. Openfactcheck: A unified framework for factuality evaluation of llms. arXiv preprint arXiv:2405.05583, 2024a. find
  30. Liu, Z. Secqa: A concise question-answering dataset for evaluating large language models in computer security, 2023. URL https://arxiv.org/abs/2312. 15838. ↗
  31. OpenAI. Gpt-4 technical report. PREPRINT, 2023.
  32. Wang, Y., Wang, M., Manzoor, M. A., Liu, F., Georgiev, G., Das, R., and Nakov, P. Factuality of large language models: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 19519–19529, 2024b.
  33. Wei, J., Karina, N., Chung, H. W., Jiao, Y. J., Papay, S., Glaese, A., Schulman, J., and Fedus, W. Measuring short-form factuality in large language models. 2024. URL https://api.semanticscholar.org/CorpusID:273877483. ↗
  34. Shen, X., Chen, Z., Backes, M., Shen, Y., and Zhang, Y. ”do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models, 2024. URL https://arxiv.org/abs/2308.03825. ↗
  35. Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.…
  36. Xu, G., Liu, J., Yan, M., Xu, H., Si, J., Zhou, Z., Yi, P., Gao, X., Sang, J., Zhang, R., Zhang, J., Peng, C., Huang, F., and Zhou, J. Cvalues: Measuring the values of chinese large language models from safety to responsibility, 2023. URL https://arxiv.org/abs/2307.09705. ↗
  37. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv: Arxiv-1809.09600, 2018.
  38. Sun, H., Zhang, Z., Deng, J., Cheng, J., and Huang, M. Safety assessment of chinese large language models. arXiv preprint arXiv:2304.10436, 2023. find
  39. Zhang, Z., Lei, L., Wu, L., Sun, R., Huang, Y., Long, C., Liu, X., Lei, X., Tang, J., and Huang, M. Safetybench: Evaluating the safety of large language models, 2024. URL https://arxiv.org/abs/2309.07045. ↗
  40. Team, G. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, 2024a. URL https://arxiv.org/abs/2403.05530. ↗
  41. Zhao, Y., Zhang, J., Chern, I., Gao, S., Liu, P., He, J., et al. Felm: Benchmarking factuality evaluation of large language models. Advances in Neural Information Processing Systems, 36, 2024. find
  42. Team, I. Internlm2 technical report, 2024b. find
  43. Team, Q. Introducing qwen1.5, February 2024c. URL https://qwenlm.github.io/blog/qwen1. 5/. ↗
  44. Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., and Duan, N. Agieval: A human-centric benchmark for evaluating foundation models, 2023.
  45. Team, Q. Qwen2.5: A party of foundation models, September 2024d. URL https://qwenlm.github.io/ blog/qwen2.5/. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tan-2024-chinese-safetyqa-safety-short/