Elamur Research⌕
☰
home / papers / 2026 / MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules

MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules

Paper · 2026 · Tong Xu, Xinzhe Cao, Zhihui Zhu et al.

Summary

Benchmarks for molecular generative models usually reward novelty, task difficulty and property matching. They rarely check whether the generated compounds might be toxic, reactive or otherwise hazardous. The authors fill this gap with MolSafeEval. Its foundation is MolSafeKG, a knowledge graph that combines hazardous-substance records from ECHA (GHS classifications) and drug-toxicity datasets with structural descriptors such as elements, functional groups and ChEMBL structural alerts. To assess a new molecule, the pipeline parses it with RDKit and retrieves the closest known hazardous compounds by Tanimoto similarity. An LLM then weighs this evidence to produce a risk score and a textual explanation. Generators are grouped into four task families: unconditional, property optimization, target protein–based and text-based. Each family has standardized datasets and protocols. The evaluator shows high accuracy on 11 safety prediction tasks. Applying it to 28 generators shows that every family can produce dangerous molecules, even when average scores look acceptable. The work therefore supports safety screening and governance of AI-driven molecular design.

Key points

  • MolSafeKG contains 83,925 compounds, 42,275 of them ECHA-listed as GHS-hazardous, and about 1.95 million triples over 11 relationship types.
  • Structural knowledge covers 117 elements, 149 functional groups (in 13 types) and 434 ChEMBL structural alerts. Safety labels span 68 GHS hazard statements in 3 classes (physical, health, environmental) and 8 drug toxicity endpoints.
  • The evaluation covers 28 models, including EDM, MolGen, MIMOSA, Pocket2Mol, TargetDiff, MolT5 and BioT5. High maximum scores and variance show that low averages can hide highly risky outputs.
  • There is a safety–function trade-off: MolGen optimizes properties strongly but yields fewer non-toxic molecules, while MIMOSA shows the reverse pattern.
  • Toxic and non-toxic outputs of the same model often have similar properties, which suggests safety can be improved without large functional loss. A stated limitation is that toxicity mechanisms absent from known hazardous molecules may go undetected.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of OxfordOxford · United Kingdom122 works · 160,277 citations🎓Zhejiang UniversityHangzhou · China224 works · 13,587 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksComputational Theory and Mathematics · 321 worksMaterials Science · 31 worksMaterials Chemistry · 31 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,341 worksBias, fairness & safety · 587 works

Cite

Cite as Tong Xu, Xinzhe Cao, Zhihui Zhu, Keyan Ding and Huajun Chen. 2026. MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1679

@inproceedings{xu2026molsafeeval,
  title = {{MolSafeEval: A Benchmark for Uncovering Safety Risks in AI-Generated Molecules}},
  author = {Tong Xu, Xinzhe Cao, Zhihui Zhu, Keyan Ding and Huajun Chen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1679},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1679}
}

References

65 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Learning, volume 202 of Proceedings of Machine Learning Research, pages 6140–6157. PMLR. find
  2. Alex GC de Sá, Yangyang Long, Stephanie Portelli, Douglas EV Pires, and David B Ascher. 2022. toxcsm: comprehensive prediction of small molecule toxicity profiles. Briefings in Bioinformatics, 23(5):bbac337. find
  3. Miriana Di Stefano, Salvatore Galati, Lisa Piazza, Carlotta Granchi, Simone Mancini, Filippo Fratini, Marco Macchia, Giulio Poli, and Tiziano Tuccinardi. 2023. Venompred 2.0: a novel in silico platform for an extended and human interpretable toxicological profiling of small molecules. Journal of Che… find
  4. Joseph L Durant, Burton A Leland, Douglas R Henry, and James G Nourse. 2002. Reoptimization of mdl keys for use in drug discovery. Journal of chemical information and computer sciences, 42(6):1273– 1280. find
  5. Carl Edwards, Tuan Lai, Kevin Ros, Garrett Honke, Kyunghyun Cho, and Heng Ji. 2022. Translation between molecules and natural language. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 375–413, Abu Dhabi, United Arab Emirates. Association for Computati…
  6. Simon Axelrod and Rafael Gomez-Bombarelli. 2022. Geom, energy-annotated molecular conformations for property prediction and molecular generation. Scientific Data, 9(1):185. find
  7. Carl Edwards, ChengXiang Zhai, and Heng Ji. 2021. Text2mol: Cross-modal molecule retrieval with natural language queries. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 595–607.
  8. Changsen Bai, Lianlian Wu, Ruijiang Li, Yang Cao, Song He, and Xiaochen Bo. 2025. Machine learning-enabled drug-induced toxicity prediction. Advanced Science, 000(000). find
  9. Peter Ertl and Ansgar Schuffenhauer. 2009. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of cheminformatics, 1:1–11. find
  10. G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins. 2012. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90–98. find
  11. Yin Fang, Ningyu Zhang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. 2024. Domain-agnostic molecular generation with chemical feedback. In ICLR. OpenReview.net. find
  12. Zhe Chen, Li Zhang, Jianqiang Sun, Rui Meng, Shuaidong Yin, and Qi Zhao. 2023. Dcamcp: A deep learning model based on capsule network and attention mechanism for molecular carcinogenicity prediction. Journal of cellular and molecular medicine, 27(20):3117–3126. find
  13. Yin Fang, Qiang Zhang, Ningyu Zhang, Zhuo Chen, Xiang Zhuang, Xin Shao, Xiaohui Fan, and Huajun Chen. 2023. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature Machine Intelligence, pages 1–12. find
  14. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. find
  15. Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B Iovanisci, Ian Snyder, and David R Koes. 2020. Three-dimensional convolutional neural networks and a cross-docked data set for structure-based drug design. Journal of chemical information and modeling, 60(9):4200–4215. find
  16. Dimitrios Christofidellis, Giorgio Giannone, Jannis Born, Ole Winther, Teodoro Laino, and Matteo Manica. 2023. Unifying molecular and textual representations via multi-task language modelling. In Proceedings of the 40th International Conference on Machine Li Fu, Shaohua Shi, Jiacai Yi, Ningning Wang… find
  17. Tianfan Fu, Wenhao Gao, Cao Xiao, Jacob Yasonik, Connor W Coley, and Jimeng Sun. 2021a. Differentiable scaffolding tree for molecular optimization. arXiv preprint arXiv:2109.10469. — [arXiv:2109.10469](https://arxiv.org/abs/2109.10469) find
  18. Tianfan Fu, Cao Xiao, Xinhao Li, Lucas M Glass, and Jimeng Sun. 2021b. Mimosa: Multi-constraint molecule sampling for molecule optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 125–133. find
  19. Wenhao Gao, Tianfan Fu, Jimeng Sun, and Connor Coley. 2022. Sample efficiency matters: a benchmark for practical molecular optimization. Advances in Neural Information Processing Systems, 35:21342– 21357. find
  20. Anna Gaulton, Louisa J. Bellis, A. Patricia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, and John P. Overington. 2011. Chembl: a large-scale bioactivity database for drug discovery. Nucleic Acids Research, 40(D1):D1100– D1107. find
  21. Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge Aguilera-Iparraguirre, Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. 2018. Automatic chemical design using a data-driven continuous representation of mol… find
  22. Haisong Gong, Qiang Liu, Shu Wu, and Liang Wang. 2024. Text-guided molecule generation with diffusion language model. Proceedings of the AAAI Conference on Artificial Intelligence, 38(1):109–117. find
  23. Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 2023. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. In International Conference on Learning Representations. find
  24. Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang, and Quanquan Gu. 2024. Decompdiff: diffusion models with decomposed priors for structure-based drug design. arXiv preprint arXiv:2403.07902. — [arXiv:2403.07902](https://arxiv.org/abs/2403.07902) find
  25. Emiel Hoogeboom, Vıctor Garcia Satorras, Clément Vignac, and Max Welling. 2022. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pages 8867–8887. PMLR. find
  26. Lei Huang, Tingyang Xu, Yang Yu, Peilin Zhao, Xingjian Chen, Jing Han, Zhi Xie, Hailong Li, Wenge Zhong, Ka-Chun Wong, and 1 others. 2024a. A dual diffusion model enables 3d molecule generation and lead optimization based on target pockets. Nature Communications, 15(1):2657. find
  27. Zhilin Huang, Ling Yang, Xiangxin Zhou, Zhilong Zhang, Wentao Zhang, Xiawu Zheng, Jie Chen, Yu Wang, Bin CUI, and Wenming Yang. 2024b. Protein-ligand interaction prior for binding-aware 3d molecule diffusion models. In International Conference on Learning Representations. find
  28. Wengong Jin, Regina Barzilay, and Tommi Jaakkola. 2018. Junction tree variational autoencoder for molecular graph generation. In International conference on machine learning, pages 2323–2332. PMLR. find
  29. Greg Landrum. 2013. Rdkit documentation. Release, 1(1-79):4. find
  30. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Proc…
  31. Jiatong Li, Yunqing Liu, Wenqi Fan, Xiao-Yong Wei, Hui Liu, Jiliang Tang, and Qing Li. 2023. Empowering molecule discovery for molecule-caption translation with large language models: A chatgpt perspective. arXiv preprint arXiv:2306.06615. — [arXiv:2306.06615](https://arxiv.org/abs/2306.06615) find
  32. Haitao Lin, Yufei Huang, Odin Zhang, Lirong Wu, Siyuan Li, Zhiyuan Chen, and Stan Z. Li. 2024a. Functional-group-based diffusion for pocket-specific molecule generation and elaboration. Preprint, arXiv:2306.13769. — [arXiv:2306.13769](https://arxiv.org/abs/2306.13769) find
  33. Haitao Lin, Guojiang Zhao, Odin Zhang, Yufei Huang, Lirong Wu, Zicheng Liu, Siyuan Li, Cheng Tan, Zhifeng Gao, and Stan Z. Li. 2024b. Cbgbench: Fill in the blank of protein-molecule complex binding graph. Preprint, arXiv:2406.10840. — [arXiv:2406.10840](https://arxiv.org/abs/2406.10840) find
  34. Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander Gaunt. 2018. Constrained graph variational autoencoders for molecule design. In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc. find
  35. Shitong Luo, Jiaqi Guan, Jianzhu Ma, and Jian Peng. 2021. A 3d generative model for structure-based drug design. In Thirty-Fifth Conference on Neural Information Processing Systems. find
  36. Natalie Maus, Haydn Jones, Juston Moore, Matt J Kusner, John Bradshaw, and Jacob Gardner. 2022. Local latent space bayesian optimization over structured inputs. Advances in neural information processing systems, 35:34505–34518. find
  37. Frederic P Miller, Agnes F Vandome, and John McBrewster. 2009. Levenshtein distance: Information theory, computer science, string (computer science), string metric, damerau? levenshtein distance, spell checker, hamming distance. find
  38. Yoochan Myung, Alex GC de Sá, and David B Ascher. 2024. Deep-pk: deep learning for small molecule pharmacokinetic and toxicity prediction. Nucleic acids research, 52(W1):W469–W475. find
  39. Matthew Ragoza, Tomohide Masuda, and David Ryan Koes. 2022. Generating 3D molecules conditional on receptor binding sites with deep generative models. Chem Sci, 13:2701–2713. find
  40. AkshatKumar Nigam, Robert Pollice, Gary Tom, Kjell Jorner, John Willes, Luca Thiede, Anshul Kundaje, and Alan Aspuru-Guzik. 2023. Tartarus: A benchmarking platform for realistic and practical inverse molecular design. In Advances in Neural Information Processing Systems, volume 36, pages 3263–3306.… find
  41. Jean-Louis Reymond. 2015. The chemical space project. Accounts of Chemical Research, 48(3):722– 730. PMID: 25687211. find
  42. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  43. Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan. 2023. Biot5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,… find
  44. Xingang Peng, Jiaqi Guan, Qiang Liu, and Jianzhu Ma. 2023. MolDiff: Addressing the atom-bond inconsistency problem in 3D molecule diffusion generation. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 27611–27629.… find
  45. Xingang Peng, Shitong Luo, Jiaqi Guan, Qi Xie, Jian Peng, and Jianzhu Ma. 2022. Pocket2mol: Efficient molecular sampling based on 3d protein pockets. In International Conference on Machine Learning. find
  46. Pedro O Pinheiro, Arian Jamasb, Omar Mahmood, Vishnu Sresht, and Saeed Saremi. 2024. Structure-based drug design by denoising voxel grids. In ICML. find
  47. Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez-Lengeling, Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Aleksey Artamonov, Vladimir Aladinskiy, Mark Veselov, Artur Kadurin, Simon Johansson, Hongming Chen, Sergey Nikolenko, Alan Aspuru-Guzik, and Alex Zhavoronkov. 2020.… find
  48. David Rogers and Mathew Hahn. 2010. Extended-connectivity fingerprints. Journal of chemical information and modeling, 50(5):742–754. find
  49. Nadine Schneider, Roger A Sayle, and Gregory A Landrum. 2015. Get your atoms in order-an open-source implementation of a novel and robust molecular canonicalization algorithm. Journal of chemical information and modeling, 55(10):2111–2120. find
  50. Arne Schneuing, Charles Harris, Yuanqi Du, Kieran Didi, Arian Jamasb, Ilia Igashov, Weitao Du, Carla Gomes, Tom L Blundell, Pietro Lio, Max Welling, Michael Bronstein, and Bruno Correia. 2024. Structure-based drug design with equivariant diffusion models. Nature Computational Science, 4(12):899–909. find
  51. Teague Sterling and John J Irwin. 2015. Zinc 15–ligand discovery for everyone. Journal of chemical information and modeling, 55(11):2324–2337. find
  52. Simon Steshin. 2023. Lo-hi: Practical ml drug discovery benchmark. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. find
  53. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. arXiv pr…
  54. Fabio Urbina, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. 2022. Dual use of artificial-intelligence-powered drug discovery. Nature machine intelligence, 4(3):189–191. find
  55. Clement Vignac, Nagham Osman, Laura Toni, and Pascal Frossard. 2023. Midi: Mixed graph and 3d denoising diffusion for molecule generation. arXiv preprint arXiv:2302.09048. — [arXiv:2302.09048](https://arxiv.org/abs/2302.09048) find
  56. Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, and Gunter Klambauer. 2018. Fréchet chemnet distance: a metric for generative models for molecules in drug discovery. Journal of chemical information and modeling, 58(9):1736– 1741. find
  57. Mingyang Wang, Zhe Wang, Huiyong Sun, Jike Wang, Chao Shen, Gaoqi Weng, Xin Chai, Honglin Li, Dongsheng Cao, and Tingjun Hou. 2022. Deep learning approaches for de novo drug design: An overview. Current Opinion in Structural Biology, 72:135–144. find
  58. Yanru Qu, Keyue Qiu, Yuxuan Song, Jingjing Gong, Jiawei Han, Mingyue Zheng, Hao Zhou, and Wei-Ying Ma. 2024. Molcraft: Structure-based drug design in continuous parameter space. find
  59. Shijie Wang, Wenqi Fan, Yue Feng, Xinyu Ma, Shuaiqiang Wang, and Dawei Yin. 2025. Knowledge graph retrieval-augmented generation for llm-based recommendation. Preprint, arXiv:2501.02226.
  60. Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. 2021. Mars: Markov molecular sampling for multi-objective drug discovery. arXiv preprint arXiv:2103.10432. — [arXiv:2103.10432](https://arxiv.org/abs/2103.10432) find
  61. Minkai Xu, Alexander Powers, Ron Dror, Stefano Ermon, and Jure Leskovec. 2023. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning. PMLR. find
  62. Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. 2022. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations. find
  63. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024. Hallucination is inevitable: An innate limitation of large language models. Preprint, arXiv:2401.11817.
  64. ZaiXi Zhang, Shuxin Zheng, Yaosen Min, and Qi Liu. 2023. Molecule generation for target protein binding with structural motifs. In International Conference on Learning Representations. find
  65. Yingying Zhu, Yanhong Zhang, Xinze Li, and Ling Wang. 2024. 3mtox: A motif-level graph-based multi-view chemical language model for toxicity identification with deep interpretation. Journal of Hazardous Materials, 476:135114. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-molsafeeval-benchmark-uncovering-safety-risks/