Elamur Research⌕
☰
home / papers / 2026 / MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics

MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics

Paper · 2026 · Yuxing Lu, Xukai Zhao, J. Ben Tamo et al.

Summary

LLMs are increasingly used in metabolomics, but nobody has systematically tested which tasks in this field they handle reliably. General biomedical benchmarks such as MedQA or BioASQ do not cover the specific operations metabolomics needs, like linking identifiers across databases or describing pathways. MetaBench fills this gap. It contains about 8,000 test items built from public resources (HMDB, KEGG, PathBank, MetaKG, MetaboLights, and ChEBI mappings via the MetabolitesID package). The items are grouped into five capability levels:

  • Knowledge: multiple-choice recall.
  • Understanding: pathway and metabolite descriptions.
  • Grounding: identifier mapping.
  • Reasoning: triple extraction.
  • Research: writing study descriptions.

The authors evaluated 25 open and proprietary LLMs, scoring classification tasks with accuracy and generation tasks with BERTScore. Models write fluent scientific text well but largely fail at cross-database identifier grounding, even when given retrieval. Accuracy also falls for poorly annotated, long-tail metabolites. The benchmark provides a standard way to compare models and shows where LLM outputs in metabolomics still need dedicated tools or human checking.

Key points

  • Dataset sizes per capability: Knowledge 2.5K, Understanding 1.3K, Grounding 1.0K, Reasoning 1.2K, Research 2.1K. The data is released on HuggingFace as LuYuxing/MetaBench.
  • The Knowledge MCQA set has 2,500 four-option questions. They cover 26 HMDB/KEGG attributes, and distractors are drawn from other values of the same attribute.
  • Grounding requires bidirectional mapping among KEGG, HMDB, ChEBI and metabolite names. Without retrieval, accuracy is below 1%; with a search API, the best result is about 41%.
  • Long-tail effect: GPT-oss-20b reaches 63.5% on well-studied metabolites but only 40.0% on rarely annotated ones, a gap of 23.5 percentage points.
  • No metabolomics-specialized LLM exists yet, so only general and biomedical models were tested. Spectra and structure modalities are left for future releases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China333 works · 87,019 citations🎓Peking UniversityBeijing · China244 works · 39,359 citations🎓Georgia Institute of TechnologyAtlanta · United States133 works · 24,497 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksEvaluation & benchmarks · 1,220 works

Cite

Cite as Yuxing Lu, Xukai Zhao, J. Ben Tamo, Micky C. Nnamdi, Rui Peng, Shuang Zeng, Xingyu Hu, Jinzhuo Wang and May Dongmei Wang. 2026. MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1506

@inproceedings{lu2026metabench,
  title = {{MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics}},
  author = {Yuxing Lu, Xukai Zhao, J. Ben Tamo, Micky C. Nnamdi, Rui Peng, Shuang Zeng, Xingyu Hu, Jinzhuo Wang and May Dongmei Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1506},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1506}
}

References

52 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925.
  3. Anthropic. 2024. Model card for claude 3. https://www-cdn.anthropic.com/ de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/ Model_Card_Claude_3.pdf. ↗
  4. Anthropic. 2025. System card: Claude opus 4 & claude sonnet 4. https://www-cdn.anthropic.com/ 4263b940cabb546aa0e3283f35b686f4f3b2ff47. pdf. ↗
  5. Madina Bekbergenova, Lucas Pradi, Benjamin Navet, Yousouf Taghzouti, Emma Tysinger, Jean-Luc Wolfender, Wout Bittremieux, Fabien Gandon, and Louis-Félix Nothias. 2025. Metabot: Ai-based agent for natural language-based interaction with metabolomics knowledge graphs. In ISMB/ECCB 2025. find
  6. Olatomiwa O Bifarin, Varun S Yelluru, Aditya Simhadri, and Facundo M Fernández. 2025. A large language model–powered map of metabolomics research. Analytical Chemistry. find
  7. Bioconductor. 2020. metaboliteidmapping: Annotation package with comprehensive mapping of metabolite ids. https://bioconductor.org/packages/ metaboliteIDmapping/. ↗
  8. John Braisted and 1 others. 2023. metlinkr: Facilitating metaanalysis of human metabolomics data through automated linking of metabolite identifiers. Bioinformatics, 39(12):btad744. find
  9. Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Changxin Wang, Zhifeng Gao, Hongshuai Wang, Yongge Li, Mujie Lin, and 1 others. 2024. Sciassess: Benchmarking llm proficiency in scientific literature analysis. arXiv preprint arXiv:2403.01976. find
  10. Tomas Cajka and Oliver Fiehn. 2024. A reproducibility crisis for clinical metabolomics studies. Trends in Analytical Chemistry, 168:117332. find
  11. Kathi Canese and Sarah Weis. 2013. Pubmed: the bibliographic database. The NCBI handbook, 2(1):2013. find
  12. Qingyu Chen, Yan Hu, Xueqing Peng, Qianqian Xie, Qiao Jin, Aidan Gilson, Maxwell B Singer, Xuguang Ai, Po-Ting Lai, Zhizheng Wang, and 1 others. 2025. Benchmarking large language models for biomedical natural language processing applications and recommendations. Nature communications, 16(1):3280. find
  13. Elizabeth A Coler, Wanxuan Chen, Alexey V Melnik, James T Morton, and Alexander A Aksenov. 2024. Metabolomics in the era of artificial intelligence. Microbiota and Host, 2(1). find
  14. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. The llama 3 herd of models. arXiv e-prints, pages arXiv–2407.
  15. Dmitry Grapov and 1 others. 2018. Consistency, inconsistency, and ambiguity of metabolite names in biochemical databases used for genome-scale metabolic modelling. Molecular BioSystems, 14(3):660–667. find
  16. Carlos Guijas, J Rafael Montenegro-Burke, Benedikt Warth, Mary E Spilker, and Gary Siuzdak. 2018. Metabolomics activity screening for identifying metabolites that modulate phenotype. Nature biotechnology, 36(4):316–320. find
  17. Yucheng Hu and Yuxing Lu. 2024. Rag and rau: A survey on retrieval-augmented language model in natural language processing. arXiv preprint arXiv:2404.19543. — [arXiv:2404.19543](https://arxiv.org/abs/2404.19543) find
  18. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
  19. Zongcheng Ji, Qiang Wei, and Hua Xu. 2020. Bert-based ranking for biomedical entity normalization. AMIA Summits on Translational Science Proceedings, 2020:269. find
  20. Jiyue Jiang, Pengan Chen, Jiuming Wang, Dongchen He, Ziqin Wei, Liang Hong, Licheng Zong, Sheng Wang, Qinze Yu, Zixian Ma, and 1 others. 2025a. Benchmarking large language models on multiple tasks in bioinformatics nlp with prompting. arXiv preprint arXiv:2503.04013. — [arXiv:2503.04013](https://arxiv.org/abs/2503.04013) find
  21. Zhuohang Jiang, Pangjing Wu, Ziran Liang, Peter Q Chen, Xu Yuan, Ye Jia, Jiancheng Tu, Chen Li, Peter HF Ng, and Qing Li. 2025b. Hibench: Benchmarking llms capability on hierarchical structure reasoning. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pa… find
  22. Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146.
  23. Adam Tauman Kalai, Ofir Nachum, Santosh S Vempala, and Edwin Zhang. 2025. Why language models hallucinate. arXiv preprint arXiv:2509.04664. — [arXiv:2509.04664](https://arxiv.org/abs/2509.04664) find
  24. Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In International conference on machine learning, pages 15696–15707. PMLR.
  25. Minoru Kanehisa, Miho Furumichi, Yoko Sato, Yuriko Matsuura, and Mari Ishiguro-Watanabe. 2025. Kegg: biological systems database as a model of the real world. Nucleic acids research, 53(D1):D672–D677. find
  26. Christopher A Krettler, Maria Sorokina, Felina Kretschmer, Daniel H Pieper, and Christoph Steinbeck. 2024. Navigating common pitfalls in metabolite identification and metabolomics bioinformatics. Metabolomics, 20(5):104. find
  27. Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis, and Georgios Paliouras. 2023. Bioasq-qa: A manually curated corpus for biomedical question answering. Scientific Data, 10(1):170. find
  28. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  29. Yijiang Liu, Feifan Zhang, Yifei Ge, Qiao Liu, Siyu He, and Xiaotao Shen. 2025. Application of llms/transformer-based models for metabolite annotation in metabolomics. Health and Metabolism, pages 7–7. find
  30. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  31. Yuxing Lu. 2025. Knowledge graph and large language model for metabolomics. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 29281–29282. find
  32. Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. 2025. A comprehensive overview of large language models. ACM Transactions on Intelligent Systems and Technology, 16(5):1–72.
  33. Rex Powers and 1 others. 2022. A checklist for reproducible computational analysis in clinical metabolomics research. Metabolites, 12(1):87. find
  34. Maxx Richard Rahman, Ruoxuan Liu, and Wolfgang Maass. 2024. Incorporating metabolic information into llms for anomaly detection in clinical time-series. arXiv preprint arXiv:2410.12830. — [arXiv:2410.12830](https://arxiv.org/abs/2410.12830) find
  35. Yuanyi Ren, Haoran Ye, Hanjun Fang, Xin Zhang, and Guojie Song. 2024. Valuebench: Towards comprehensively evaluating value orientations and understanding of large language models. arXiv preprint arXiv:2406.04214.
  36. Ute Roessner and Jairus Bowne. 2009. What is metabolomics all about? Biotechniques, 46(5):363– 365. find
  37. Neil Swainston, Kieran Smallbone, Pedro Mendes, Douglas Kell, and Norman Paton. 2014. Comparative evaluation of open source software for mapping between metabolite identifiers in metabolic network reconstructions: application to recon 2. Journal of Cheminformatics, 6(1):1–11. find
  38. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. find
  39. Benyou Wang, Qianqian Xie, Jiahuan Pei, Zhihong Chen, Prayag Tiwari, Zhao Li, and Jie Fu. 2023. Pre-trained language models in biomedical domain: A systematic survey. ACM Computing Surveys, 56(3):1– 52. find
  40. Pengkun Wang, Zhe Zhao, HaiBin Wen, Fanfu Wang, Binwu Wang, Qingfu Zhang, and Yang Wang. 2024. Llm-autoda: Large language model-driven automatic data augmentation for long-tailed problems. Advances in Neural Information Processing Systems, 37:64915–64941. find
  41. Wolfram Weckwerth. 2003. Metabolomics in systems biology. Annual review of plant biology, 54(1):669– 689. find
  42. Leigh Weston, Vahe Tshitoyan, John Dagdelen, Olga Kononova, Amalie Trewartha, Kristin A Persson, Gerbrand Ceder, and Anubhav Jain. 2019. Named entity recognition and normalization applied to large-scale information extraction from the materials science literature. Journal of chemical information and… find
  43. Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. 2018. Moleculenet: a benchmark for molecular machine learning. Chemical science, 9(2):513–530. find
  44. Yinglin Xia and Jun Sun. 2022. Statistical data analysis of microbiomes and metabolomics, volume 13. American Chemical Society. find
  45. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  46. Ozgur Yurekten, Thomas Payne, Noemi Tejera, Felix Xavier Amaladoss, Callum Martin, Mark Williams, and Claire O’Donovan. 2024. Metabolights: open data repository for metabolomics. Nucleic acids research, 52(D1):D640–D646. find
  47. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  48. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2).
  49. Xukai Zhao, He Huang, Tao Yang, Yuxing Lu, Lu Zhang, Ruoyu Wang, Zhengliang Liu, Tianyang Zhong, and Tianming Liu. 2025. Urban planning in the age of large language models: Assessing openai o1’s performance and capabilities across 556 tasks. Computers, Environment and Urban Systems, 121:102332. find
  50. David S Wishart, AnChi Guo, Eponine Oler, Fei Wang, Afia Anjum, Harrison Peters, Raynard Dizon, Zinat Sayeeda, Siyang Tian, Brian L Lee, and 1 others. 2022. Hmdb 5.0: the human metabolome database for 2022. Nucleic acids research, 50(D1):D622– D631. find
  51. David S Wishart, Ray Kruger, Aadhavya Sivaku-maran, Karxena Harford, Selena Sanford, Rahil Doshi, Nitya Khetarpal, Omolola Fatokun, Daph-nee Doucet, Ashley Zubkowski, and 1 others. 2024. Pathbank 2.0—the pathway database for model organism metabolomics. Nucleic acids research, 52(D1):D654–D662. find
  52. Gert Wohlgemuth, Pradeep Kumar Haldiya, Egon Willighagen, Tobias Kind, and Oliver Fiehn. 2010. The find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lu-2026-metabench-multi-task-benchmark-assessing/