Elamur Research⌕
☰
home / papers / 2024 / DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration

DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration

Paper · 2024 · Sizhe Liu, Yizhou Lu, Siyu Chen et al.

Summary

DrugAgent targets a practical gap in AI-aided drug discovery: LLMs can write general ML code, but they often stumble on domain details. Examples include handling SMILES strings, choosing the right biology/chemistry libraries, or encoding proteins. General agents such as ReAct and ResearchAgent lack this expertise, while chemistry tool-agents like ChemCrow do not support full ML pipelines.

The authors propose a two-agent system built on GPT-4o:

  • LLM Planner: proposes several candidate solution ideas, sends them out for testing, and prunes weak or infeasible ones based on the results.
  • LLM Instructor: checks where domain knowledge is needed, builds specialized tools from a curated documentation library, writes and runs the code, and reports success or failure.

The documentation library covers data acquisition, featurization and pretrained domain models. The system was tested on three binary classification tasks: ADMET (PAMPA), HTS (HIV) and DTI (DAVIS). DrugAgent produced more valid submissions than the general-purpose agents and came close to or matched expert-chosen baselines. On DTI it gained 4.92% relative ROC-AUC over ReAct. The work suggests that explicitly injecting domain knowledge makes automated ML programming in specialized sciences more reliable.

Key points

  • Results on DAVIS (DTI): DrugAgent reached ROC-AUC 0.8950 with an 87.5% valid rate. This compares with ReAct (0.8530, 50%), ResearchAgent (0.8793, 75%) and the Human baseline (0.8940). CoT produced no valid runs.
  • Results on PAMPA and HIV: DrugAgent@Top3 scored 0.8206 and 0.8257 ROC-AUC, against Human baselines of 0.8173 and 0.8305, with a 100% valid rate on both. DrugAgent@Top1 was weaker on PAMPA (0.7667), which the authors attribute to a mismatch between validation and test performance.
  • Ablation on DTI: removing the Planner lowered ROC-AUC to 0.8845, and removing the Instructor lowered it to 0.8770, with the valid rate also dropping to 75%.
  • Documentation library: it covers raw data retrieval, molecule and protein featurization (fingerprints, graphs), and foundation models such as ChemBERTa and ESM.
  • Evaluation protocol: each method had eight runs. A run counted as valid only if the code ran bug-free, the output was correctly formatted, and the score was no more than 10% below the human baseline. Trace analysis showed no domain-knowledge errors for DrugAgent, unlike ReAct and ResearchAgent.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States567 works · 335,190 citations🎓Carnegie Mellon UniversityPittsburgh · United States633 works · 195,218 citations🎓University of Southern CaliforniaLos Angeles · United States183 works · 37,526 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksComputational Theory and Mathematics · 496 worksMaterials Science · 58 worksMaterials Chemistry · 58 works

Cite

Cite as Sizhe Liu, Yizhou Lu, Siyu Chen, Xiyang Hu, Jieyu Zhao, Yingzhou Lu and Yue Zhao. 2024. DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration. arXiv:2411.15692.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2411.15692 · arXiv 2411.15692

@misc{liu2024drugagent,
  title = {{DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration}},
  author = {Sizhe Liu, Yizhou Lu, Siyu Chen, Xiyang Hu, Jieyu Zhao, Yingzhou Lu and Yue Zhao},
  year = {2024},
  doi = {10.48550/arxiv.2411.15692},
  eprint = {2411.15692},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2411.15692}
}

References

36 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. CBDD Group. 2020. Pybiomed: A toolkit for calculating molecular descriptors and analyzing biological molecules. Accessed: February 13, 2025. find
  2. Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, et al. 2024b. Position: Trustllm: Trustworthiness in large language models. In International Conference on Machine Learning, pages 20166–20270. PMLR.
  3. Jintai Chen, Yaojun Hu, Yue Wang, Xu Cao, Miao Lin, Hongxia Xu, Jian Wu, Cao Xiao, Jimeng Sun, et al. 2024a. Trialbench: Multi-modal artificial intelligence-ready clinical trial datasets. arXiv:2407.00631. — [arXiv:2407.00631](https://arxiv.org/abs/2407.00631) find
  4. Yingzhou Lu, Chiung-Ting Wu, Sarah J Parker, Zuolin Cheng, Georgia Saylor, Jennifer E Van Eyk, Guoqiang Yu, Robert Clarke, David M Herrington, and Yue Wang. 2022. COT: an efficient and accurate method for detecting marker genes among many subtypes. Bioinformatics Advances, 2(1):vbac037. find
  5. Tatsuro Inaba, Hirokazu Kiyomaru, Fei Cheng, and Sadao Kurohashi. 2023. MultiTool-CoT: GPT-3 can use multiple external tools with chain of thought prompting. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1522–1532, Toronto,… find
  6. Jakub Lála, Odhran O’Donoghue, Aleksandar Shtedritski, Sam Cox, Samuel G. Rodriques, and Andrew D. White. 2023. Paperqa: Retrieval-augmented generative agent for scientific research. arXiv preprint arXiv:2312.07559.
  7. Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515.
  8. Greg Landrum. 2023. Rdkit: Open-source cheminformatics. find
  9. Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. 2024. Augmenting large language models with chemistry tools. Nature Machine Intelligence, 6(5):525–535. find
  10. Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, and Yixin Wang. 2024a. Mmedagent: Learning to use medical tools with multi-modal agent. arXiv preprint arXiv:2407.02483. Accepted at EMNLP 2024. find
  11. Zhangming Niu, Xianglu Xiao, Wenfan Wu, Qiwei Cai, Yinghui Jiang, Wangzhen Jin, Minhao Wang, Guo-jian Yang, Lingkang Kong, Xurui Jin, Guang Yang, and Hongming Chen. 2024. Pharmabench: Enhancing admet benchmarks with large language models. Scientific Data, 11(985). find
  12. Mufei Li, Jinjing Zhou, Jiajing Hu, Wenxuan Fan, Yangkang Zhang, Yaxin Gu, and George Karypis. 2021. Dgl-lifesci: An open-source toolkit for deep learning on graphs in life science. ACS Omega, 6(41):27233–27238. find
  13. Sylwia Nowakowska. 2023. Chemberta-2: Fine-tuning for molecule’s HIV replication inhibition prediction. find
  14. Ziming Li, Qianbo Zang, David Ma, Jiawei Guo, Tuney Zheng, Minghao Liu, Xinyao Niu, Yue Wang, Jian Yang, Jiaheng Liu, Wanjun Zhong, Wangchunshu Zhou, Wenhao Huang, and Ge Zhang. 2024b. Autokaggle: A multi-agent framework for autonomous data science competitions. arXiv preprint arXiv:2410.20424. find
  15. OpenAI. 2024. Chatgpt: Gpt-4o-2024-08-06. find
  16. Thai-Hoang Pham, Yue Qiu, Jucheng Zeng, Lei Xie, and Ping Zhang. 2021. A deep learning framework for high-throughput mechanism-driven phenotype compound screening and its application to covid-19 drug repurposing. Nature Machine Intelligence, 3(3):247–257. find
  17. Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. 2022. Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv. find
  18. Sudeep Pushpakom, Francesco Iorio, Patrick A Eyers, K Jane Escott, Shirley Hopper, Andrew Wells, Andrew Doig, Tim Guilliams, Joanna Latimer, Christine McNamee, et al. 2019. Drug repurposing: progress, challenges and recommendations. Nature Reviews Drug Discovery, 18(1):41–58. find
  19. Sizhe Liu, Yuchen Liu, Haofeng Xu, Jun Xia, and Stan Z. Li. 2025. Sp-dti: Subpocket-informed transformer for drug-target interaction prediction. Bioinformatics, btaf011. find
  20. Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu, Z… — Tool Learning with Foundation Models
  21. Sizhe Liu, Jun Xia, Lecheng Zhang, Yuchen Liu, Yue Liu, Wenjie Du, Zhangyang Gao, Bozhen Hu, Cheng Tan, Hongxin Xiang, and Stan Z. Li. 2024. Flex-mol: A flexible toolkit for benchmarking molecular relational learning. In Proceedings of the 38th Conference on Neural Information Processing Systems (Ne… find
  22. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. 2024a. The AI Scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292.
  23. Chidaksh Ravuru, Sagar Srinivas Sakhinana, and Venkataramana Runkana. 2024. Agentic retrieval-augmented generation for time series analysis. In Proceedings of the Undergraduate Consortium at ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). find
  24. Yingzhou Lu, Tianyi Chen, Nan Hao, Capucine Van Rechem, Jintai Chen, and Tianfan Fu. 2024b. Uncertainty quantification and interpretability for clinical trial approval prediction. Health Data Science, 4:0126. find
  25. Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C. Lawrence Zitnick, Jerry Ma, and Rob Fergus. 2019. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS. find
  26. Zheng, Siyuan Li, and Stan Z. Li. 2023. Understanding the limitations of deep models for molecular property prediction: Insights and solutions. In NeurIPS 2023. Last Modified: 03 Nov 2023. find
  27. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).
  28. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Preprint, arXiv:2302.04761.
  29. Sion Yoon, Tae Eun Kim, and Yoo Jung Oh. 2024. Designing and evaluating multi-chatbot interface for human-ai communication: Preliminary findings from a persuasion task. arXiv preprint arXiv:2406.19648. — [arXiv:2406.19648](https://arxiv.org/abs/2406.19648) find
  30. V. B. Siramshetty, P. Shah, et al. 2021. Validating adme qsar models using marketed drugs. SLAS Discovery, 26(10):1326–1336. find
  31. Ling Yue, Sixue Xing, Jintai Chen, and Tianfan Fu. 2024. Clinicalagent: Clinical trial multi-agent with large language model-based reasoning. arXiv preprint arXiv:2404.14777. — [arXiv:2404.14777](https://arxiv.org/abs/2404.14777) find
  32. Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean Hendryx, Summer Yue, and Hugh Zhang. 2024a. Planning in natural language improves llm search for code generation. arXiv preprint arXiv:2409.03733. find
  33. Bai Zhang, Yi Fu, Yingzhou Lu, Zhen Zhang, Robert Clarke, Jennifer E Van Eyk, David M Herrington, and Yue Wang. 2021. DDN2.0: R and python packages for differential dependency network analysis of biological systems. bioRxiv, pages 2021–04. find
  34. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, and et al. 2024b. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6).
  35. Hakime Öztürk, Arzucan Özgür, and Elif Ozkirimli. 2018. Deepdta: Deep drug–target binding affinity prediction. Bioinformatics, 34(17):i821–i829. find
  36. Yue Wang, Tianfan Fu, Yinlong Xu, Zihan Ma, Hongxia Xu, Bang Du, Honghao Gao, Jian Wu, and Jintai Chen. 2024c. Twin-gpt: Digital twins for clinical trials via large language model. ACM Transactions on Multimedia Computing, Communications and Applications. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2024-drugagent-automating-ai-aided-drug/