Elamur Research⌕
☰
home / papers / 2026 / Automated Analysis of Global AI Safety Initiatives: A Taxonomy-Driven LLM Approach

Automated Analysis of Global AI Safety Initiatives: A Taxonomy-Driven LLM Approach

Paper · 2026 · Takayuki Semitsu, Naoto Kiribuchi and Kengo Zenitani

Summary

AI safety policy documents from governments, international bodies and standards organizations differ widely in structure, vocabulary and granularity, and existing crosswalks usually rely on one-off comparison axes. The authors, from the Japan AI Safety Institute, propose an automated, LLM-driven crosswalk that compares two documents along a fixed, reusable taxonomy: the 15 activity categories of the Activity Map on AI Safety (AMAIS). AMAIS is grounded in the Hiroshima AI Process and the Seoul Declaration.

For each category, the system extracts and maps activities from both documents. It then writes a short summary for each side, describes their commonalities and differences, and assigns a 0–5 similarity score.

The framework was applied to ten public documents, using five LLMs and Japanese prompts and outputs. Stability across models was analysed through mean, standard-deviation and model-pair MAD heatmaps, and validity was checked against expert judgments. The results show that model choice strongly shapes the outcomes, and some document pairs provoke large cross-model disagreement. Experts agreed closely with one another but diverged from model scores. The tool is meant to support inspection of policy documents, not to rank them.

Key points

  • Shared taxonomy: the aspect set is AMAIS activities p1–p15, for example Content Authentication and Provenance, AI Safety Evaluation and Red-Teaming, and International Coordination and Cooperation.
  • Data: ten documents (A–J) form nine pairs, each anchored on Document A, the UK AISI "Our Research Agenda."
  • Pipeline: activity extraction and mapping were precomputed with ChatGPT-5.2 and held fixed. Five LLMs then generated comparisons under identical prompts and temperature.
  • Scoring: the six-level scale runs from 0 (mostly different) to 5 (nearly identical). A score of 0 is forced when either document has no item mapped to a category. Outputs are strict JSON with evidence excerpts, page numbers and confidence values.
  • Validation: three experts annotated two document pairs, showing high inter-annotator agreement but a notable gap from LLM-assigned scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSocial Sciences · 1,954 worksSafety Research · 626 worksPolitical Science and International Relations · 206 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksBias, fairness & safety · 587 works

Cite

Cite as Takayuki Semitsu, Naoto Kiribuchi and Kengo Zenitani. 2026. Automated Analysis of Global AI Safety Initiatives: A Taxonomy-Driven LLM Approach. Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4yuq3ezxohee

@inproceedings{semitsu2026automated,
  title = {{Automated Analysis of Global AI Safety Initiatives: A Taxonomy-Driven LLM Approach}},
  author = {Takayuki Semitsu, Naoto Kiribuchi and Kengo Zenitani},
  booktitle = {Proceedings of the 3rd Workshop on Natural Language Processing for Political Sciences (PoliticalNLP 2026)},
  year = {2026},
  doi = {10.63317/4yuq3ezxohee},
  url = {https://doi.org/10.63317/4yuq3ezxohee}
}

References

25 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. ASEAN. 2024. Asean guide on ai governance and ethics. PDF. find
  2. Blair Attard-Frost, Ana Brandusescu, and Kelly Lyons. 2024. The governance of artificial intelligence in canada: Findings and opportunities from a review of 84 ai governance initiatives. Government Information Quarterly, 41(2):101929. find
  3. Amna Batool, Sunny Lee, Yue Liu, and Liming Dong. 2025. The anatomy of AI policies: a systematic comparative analysis of AI policies across the globe. AI and Ethics. find
  4. Catarina G Belém, Pouya Pezeshkpour, Hayate Iso, Seiji Maekawa, Nikita Bhutani, and Estevam Hruschka. 2025. From single to multi: How LLMs hallucinate in multi-document summarization. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 5291–5324, Albuquerque, New Mexico.… find
  5. Yoshua Bengio, Tegan Maharaj, Luke Ong, Stuart Russell, Dawn Song, Max Tegmark, Lan Xue, Ya-Qin Zhang, Stephen Casper, Wan Sie Lee, et al. 2025. The singapore consensus on global ai safety research priorities. arXiv preprint arXiv:2506.20702. — [arXiv:2506.20702](https://arxiv.org/abs/2506.20702) find
  6. CIFAR. 2026. CAISI Research Program at CIFAR 2025 Year in Review: Building Safe AI for Canadians. PDF. find
  7. Government of France. 2025. National strategic review 2025. PDF. find
  8. Beliz Gunel, Sandeep Tata, and Marc Najork. 2023. Strum: Extractive aspect-based contrastive summarization. In Companion Proceedings of the ACM Web Conference 2023, WWW ’23 Companion, page 28–31, New York, NY, USA. Association for Computing Machinery. find
  9. Hamza Harkous, Kassem Fawaz, Rémi Lebret, Florian Schaub, Kang G. Shin, and Karl Aberer. 2018. Polisis: Automated analysis and presentation of privacy policies using deep learning. In 27th USENIX Security Symposium (USENIX Security 18), pages 531–548, Baltimore, MD. USENIX Association. find
  10. Hayate Iso, Xiaolan Wang, Stefanos Angelidis, and Yoshi Suhara. 2022. Comparative opinion summarization via collaborative decoding. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3307–3324. find
  11. Japan AI Safety Institute. 2025a. AMAIS: Activity Map on AI Safety. PDF. find
  12. Japan AI Safety Institute. 2025b. National status report of AI Safety in Japan 2024. and Enrique Herrera-Viedma. 2022. Summarizing legal regulatory documents using transformers. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR… find
  13. MIC. 2023. Hiroshima AI Process. Web page. find
  14. OECD. 2025. Governing with artificial intelligence. find
  15. Joykirat Singh, Sehban Fazili, Rohan Jain, and Md. Shad Akhtar. 2024. EROS:entity-driven controlled policy document summarization. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 6236–6246, Torino, It… find
  16. South Korean Ministry of Government Legislation. 2025. Framework act on the development of artificial intelligence and establishment of trust (translated draft). Translated by Etcetera Language Group, Inc. find
  17. Thomas Ströhle, Ricardo Campos, and Adam Jatowt. 2024. Contrastive text summarization: a survey. International Journal of Data Science and Analytics, 18(4):353–367. find
  18. AI Seoul Summit. 2024. Seoul declaration for safe, innovative and inclusive ai. PDF. find
  19. The White House (US). 2023. Safe, secure, and trustworthy development and use of artificial intelligence. Web page. find
  20. The White House (US). 2025. America’s action plan. PDF. find
  21. Autumn Toney, Kathleen Curlee, and Emelia Probasco. 2024. Trust issues: Discrepancies in trustworthy ai keywords use in policy and research. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’24, page 2222–2233, New York, NY, USA. Association for Computin… find
  22. Mariia Tytarenko, Tobias Walter Rutar, Stefan Lengauer, and Tobias Schreck. 2025. Llm-agent support for two-document comparison using hierarchical topic maps. In VISxGenAI Workshop Papers. find
  23. UK-AISI. 2025. Our research agenda. Web page. find
  24. Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines. Nature Machine Intelligence. find
  25. Svea Klaus, Ria Van Hecke, Kaweh Djafari Naini, Ismail Sengor Altingovde, Juan Bernabé-Moreno, Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, et al. 2023. Survey on factuality in large language models: Knowledge, ret…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/semitsu-2026-automated-analysis-global-ai-safety/