Elamur Research⌕
☰
home / papers / 2026 / AnswerCarefully: Creating a Dataset for LLM Safety in Japanese

AnswerCarefully: Creating a Dataset for LLM Safety in Japanese

Summary

Most safety alignment and red-teaming for large language models is done in English. What counts as harmful, however, depends on local culture, taboos, prejudices and crime patterns, and non-English prompts can expose additional weaknesses. Japan had no broad-coverage safety resource, so the authors built AnswerCarefully. It is a hand-written collection of risky Japanese questions, each paired with a carefully composed reference answer. The questions follow the risk taxonomy of the Do-Not-Answer (DNA) dataset, but every sample was written from scratch to reflect how LLMs are actually used in Japanese society. The reference answers were shaped by the principles of the G7 Hiroshima AI Process. They let the data serve both as an evaluation benchmark and as instruction-tuning material, and they also help automatic evaluation. Fine-tuning a Japanese LLM on the data made its outputs safer while preserving general helpfulness. The authors also used the test set to rate 12 Japanese LLMs on a five-point safety scale. They argue that region-specific safety data matter and add meta tags to help others build similar datasets in other languages and regions.

Key points

  • Size and splits: AnswerCarefully (Version 2.2, May 2025; v1.0 appeared April 2024) has 1,800 question–answer pairs. The development set holds 1,464 items and the test set holds 336 (6 for each of 56 Level-3 categories).
  • Taxonomy: It follows DNA's three-level hierarchy of 5 risk areas and 12 harm types. The 61 specific harms were reduced to 56 to balance sample counts. The categories are mapped against AILuminate v1.0 and Llama Guard.
  • Authorship: About 20 human writers from varied backgrounds produced the samples. DNA, by contrast, used GPT-4 generation, which can feel templatic. The data target everyday misuse rather than expert-level attacks.
  • Evaluation and judge agreement: Human judges applied a two-step, five-level rubric (harmful 1–2 vs. safe 3–5). Of 2,196 ratings (12 systems × 183 questions), 169 disagreed across the safe/unsafe boundary. The most frequent cause was how to treat false information (23.1%).
  • Licensing: The dataset is released openly. Its only condition is that it be used to improve LLM safety, and commercial use is allowed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/289xxmwcworn

@inproceedings{suzuki2026answercarefully,
  title = {{AnswerCarefully: Creating a Dataset for LLM Safety in Japanese}},
  author = {Suzuki, Hisami and Katsumata, Satoru and Kodama, Takashi and Takahashi, Tetsuro and Nakayama, Kouta and Sekine, Satoshi},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/289xxmwcworn},
  url = {https://doi.org/10.63317/289xxmwcworn}
}

References

24 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aakanksha, Arash Ahmadian, Beyza Ermis, Seraphina Goldfarb-Tarrant, Julia Kreutzer, Marzieh Fadaee, and Sara Hooker. 2024. The multilingual alignment prism: Aligning global and local preferences to reduce harm. ArXiv: 2406.18682. find
  2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… find
  3. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared… find
  4. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. 2022b. Constitutional AI: Harmlessness from AI Feedback. ArXiv: 2212.08073. find our card
  5. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2024. Multilingual jailbreak challenges in large language models. In The Twelfth International Conference on Learning Representations. find our card
  6. Felix Friedrich, Simone Tedeschi, Patrick Schramowski, Manuel Brack, Roberto Navigli, Huu Nguyen, Bo Li, and Kristian Kersting. 2025. LLMs lost in translation: M-ALERT uncovers cross-linguistic safety inconsistencies. ArXiv: 2412.15035. — [arXiv:2412.15035](https://arxiv.org/abs/2412.15035) find
  7. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356–3369. Association for Computational Linguistics. find our card
  8. Gemini. 2024. Gemini: A family of highly capable multimodal models. ArXiv:2312.11805. find our card
  9. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:… find our card
  10. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. Llama Guard: LLM-based input-output safeguard for human-AI conversations. ArXiv: 2312.06674. find our card
  11. Satoru Katsumata, Takashi Kodama, and Yusuke Miyao. 2025. Investigating tuning methods for achieving usefulness and safety for Japanese large language models (in Japanese). In Proceedings of The 31st Annual Conference of the Association for Natural Language Processing. find
  12. Akito Nakanishi, Yukie Sano, Geng Liu, and Francesco Pierri. 2025. Analyzing the safety of japanese large language models in stereotype-triggering prompts. ArXiv: 2503.01947. — [arXiv:2503.01947](https://arxiv.org/abs/2503.01947) find
  13. Tomoka Nakazato, Masaki Onishi, Hisami Suzuki, and Yuya Shibuya. 2024. JSocialFact: a misinformation dataset from social media for benchmarking LLM safety. In 2024 IEEE International Conference on Big Data (BigData), pages 3017– 3025. find
  14. Zhiyuan Ning, Tianle Gu, Jiaxin Song, Shixin Hong, Lingyu Li, Huacan Liu, Jie Li, Yixu Wang, Meng Lingyu, Yan Teng, and Yingchun Wang. 2025. LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models. ArXiv: 2508.12733. — [arXiv:2508.12733](https://arxiv.org/abs/2508.12733) find
  15. OpenAI. 2024. GPT-4 technical report. ArXiv: 2303.08774. find our card
  16. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105. Association for Compu… find our card
  17. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… find
  18. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. 2024a. DecodingTrust: A Comprehensive Assessment… find our card
  19. Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao, and Michael Lyu. 2024b. All languages matter: On the multilingual safety of LLMs. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5865–5877. Association for Computational Linguistics. find
  20. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2024c. Do-Not-Answer: Evaluating safeguards in LLMs. In Findings of the Association for Computational Linguistics: EACL 2024, pages 896–911, St. Julian’s, Malta. Association for Computational Linguistics. find
  21. Yuxia Wang, Zenan Zhai, Haonan Li, Xudong Han, Shom Lin, Zhenxuan Zhang, Angela Zhao, Preslav Nakov, and Timothy Baldwin. 2024d. A Chinese Dataset for Evaluating the Safeguards in Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 3106–3119, Bangkok,… find
  22. Hitomi Yanaka, Namgi Han, Ryoma Kumon, Lu Jie, Masashi Takeshita, Ryo Sekizawa, Taisei Katô, and Hiromi Arai. 2025. JBBQ: Japanese bias benchmark for analyzing social biases in large language models. In Proceedings of the 6th Workshop on Gender Bias in Natural Language Processing (GeBNLP), Vienna, A… find
  23. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-judge with MT-bench and Chatbot Arena. In Advances in Neural Information Processing Systems, volu… find
  24. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. ArXiv: 2307.15043. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/suzuki-2026-answercarefully-creating-dataset-llm-safety/