Elamur Research⌕
☰
home / papers / 2026 / ClimAgent: LLM as Agents for Autonomous Open-ended Climate Science Analysis

ClimAgent: LLM as Agents for Autonomous Open-ended Climate Science Analysis

Paper · 2026 · Hao Wang, Jindong Han, Wei Fan et al.

Summary

Most prior work on LLMs in climate science tests question answering. Real climate research is different: it requires turning a loosely stated scenario into a physically grounded, data-driven analysis. ClimAgent is an autonomous agent built for this open-ended setting. Its foundation is the Climate Environment (CE), a shared action space assembled by surveying hundreds of climate papers across 25 sub-fields. CE gives the agent specialized tools, datasets and equations. The agent works like a domain expert, in four stages:

  1. It breaks the problem into sub-tasks.
  2. It retrieves the relevant modeling knowledge and refines each sub-task so that physical constraints are kept.
  3. It writes and runs code to compute the results.
  4. It compiles the results into a LaTeX report.

To evaluate such systems, the authors also release ClimaBench, a benchmark of real climate events from 2000–2025. ClimAgent beats strong agent baselines on ClimaBench, and its scores from automatic judging closely match ratings from human experts. Together, the agent and benchmark give a reproducible starting point for automating climate research.

Key points

  • CE combines 150 climate tools and 30 databases. Its data comes from public sources such as NOAA, ERA5 and NASA Earthdata, mostly under CC-BY or open-government licenses.
  • ClimaBench has 220 open-ended problems in 5 task categories. Solving them requires interpreting the problem, formulating a model and doing numerical reasoning.
  • ClimAgent improves solution rigor and practicality by 40.21% over plain LLM solutions, at roughly half the token cost of competing state-of-the-art approaches.
  • Ablation studies take away three components: the Climate Environment (CE), Problem Analysis (PA) and Climate Modeling (CM). Each removal lowers performance; dropping PA especially hurts AE and MR. In a human check, five climate PhD students rated 100 solutions, and their scores differed from the automatic ones by at most 0.08 per metric.
  • The system mainly relies on large backbones (GPT-4o, Qwen3, DeepSeek-R1). The smaller Qwen2.5-72B followed instructions less reliably and produced less rigorous models.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hong Kong University of Science and TechnologyHong Kong · China142 works · 25,136 citations🎓The Hong Kong University of Science and Technology (Guangzhou)Guangzhou · China53 works · 1,367 citations🎓University of AucklandAuckland · New Zealand11 works · 888 citations🎓Shandong UniversityJinan · China17 works · 789 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Hao Wang, Jindong Han, Wei Fan and Hao Liu. 2026. ClimAgent: LLM as Agents for Autonomous Open-ended Climate Science Analysis. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1067

@inproceedings{wang2026climagent,
  title = {{ClimAgent: LLM as Agents for Autonomous Open-ended Climate Science Analysis}},
  author = {Hao Wang, Jindong Han, Wei Fan and Hao Liu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1067},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1067}
}

References

40 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609.
  3. Sebastian Bathiany, Vasilis Dakos, Marten Scheffer, and Timothy M Lenton. 2018. Climate models predict increasing temperature variability in poor countries. Science advances, 4(5):eaar5809. find
  4. Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. 2023. Autonomous chemical research with large language models. Nature, 624(7992):570– 578.
  5. Tamma A Carleton and Solomon M Hsiang. 2016. Social and economic impacts of climate. Science, 353(6304):aad9837. find
  6. Jingyi Chai, Shuo Tang, Rui Ye, Yuwen Du, Xinyu Zhu, Mengcheng Zhou, Yanfeng Wang, Yuzhi Zhang, Linfeng Zhang, Siheng Chen, and 1 others. 2025. Scimaster: Towards general-purpose scientific ai agents, part i. x-master as foundation: Can we lead on humanity’s last exam? arXiv preprint arXiv:2507.0524…
  7. Natalia De La Calzada, Théo Alves Da Costa, Annabelle Blangero, and Nicolas Chesneau. 2024. Climateq&a: Bridging the gap between climate scientists and the general public. arXiv preprint arXiv:2403.14709. — [arXiv:2403.14709](https://arxiv.org/abs/2403.14709) find
  8. Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, and Jun Wang. 2024. Ds-agent: Automated data science by empowering large language models with case-based reasoning. arXiv preprint arXiv:2402.17453. find
  9. Jindong Han, Hao Wang, Hui Xiong, and Hao Liu. 2025. Scalable pre-training of compact urban spatiotemporal predictive models on large-scale multi-domain data. Proceedings of the VLDB Endowment, 18(7):2149–2158. find
  10. J Hansen, R Ruedy, A Lacis, Mki Sato, L Nazarenko, N Tausnev, I Tegen, and D Koch. 2000. Climate modeling in the global warming debate. In International Geophysics, volume 70, pages 127–164. Elsevier. find
  11. Caroline R Holmes, Tim Woollings, Ed Hawkins, and Hylke De Vries. 2016. Robust future changes in temperature variability under greenhouse gas forcing and the relationship with thermal advection. Journal of Climate, 29(6):2221–2236. find
  12. Frédéric Hourdin, Thorsten Mauritsen, Andrew Get-telman, Jean-Christophe Golaz, Venkatramani Balaji, Qingyun Duan, Doris Folini, Duoying Ji, Daniel Klocke, Yun Qian, and 1 others. 2017. The art and science of climate model tuning. Bulletin of the American Meteorological Society, 98(3):589–602. find
  13. Ming Hu, Chenglong Ma, Wei Li, Wanghan Xu, Jiamin Wu, Jucheng Hu, Tianbin Li, Guohang Zhuang, Jiaqi Liu, Yingzhou Lu, and 1 others. 2025a. A survey of scientific large language models: From data foundations to agent frontiers. arXiv preprint arXiv:2508.21148. — [arXiv:2508.21148](https://arxiv.org/abs/2508.21148) find
  14. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, and 1 others. 2025b. Memory in the age of ai agents. arXiv preprint arXiv:2512.13564. — [arXiv:2512.13564](https://arxiv.org/abs/2512.13564) find
  15. Kexin Huang, Serena Zhang, Hanchen Wang, Yuanhao Qu, Yingzhou Lu, Yusuf Roohani, Ryan Li, Lin Qiu, Gavin Li, Junze Zhang, and 1 others. 2025. Biomni: A general-purpose biomedical ai agent. biorxiv. find
  16. Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2023. Benchmarking large language models as ai research agents. In NeurIPS 2023 Foundation Models for Decision Making Workshop. find
  17. Raj Jaiswal, Dhruv Jain, Harsh Parimal Popat, Avinash Anand, Abhishek Dharmadhikari, Atharva Marathe, and Rajiv Ratn Shah. 2024. Improving physics reasoning in large language models using mixture of refinement agents. arXiv preprint arXiv:2412.00821. find
  18. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770.
  19. Ruofan Jin, Zaixi Zhang, Mengdi Wang, and Le Cong. 2025. Stella: Self-evolving llm agent for biomedical research. arXiv preprint arXiv:2507.02004. find
  20. Maximilian Kotz, Anders Levermann, and Leonie Wenz. 2024. The economic commitment of climate change. Nature, 628(8008):551–557. find
  21. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  22. Fan Liu, Zherui Yang, Cancheng Liu, Tianrui Song, Xiaofeng Gao, and Hao Liu. 2025. Mm-agent: Llm as agents for real-world mathematical modeling problem. arXiv preprint arXiv:2505.14148. — [arXiv:2505.14148](https://arxiv.org/abs/2505.14148) find
  23. Veeramakali Vignesh Manivannan, Yasaman Jafari, Srikar Eranky, Spencer Ho, Rose Yu, Duncan Watson-Parris, Yian Ma, Leon Bergen, and Taylor Berg-Kirkpatrick. 2024. Climaqa: An automated evaluation framework for climate question answering models. arXiv preprint arXiv:2410.16701. — [arXiv:2410.16701](https://arxiv.org/abs/2410.16701) find
  24. Friederike EL Otto. 2017. Attribution of weather and climate events. Annual Review of Environment and Resources, 42:627–646. find
  25. Camille Parmesan, Mike D Morecroft, and Yongyut Trisurat. 2022. Climate change 2022: Impacts, adaptation and vulnerability. Ph.D. thesis, GIEC. find
  26. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. 2025. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227.
  27. Sonia I Seneviratne, Xuebin Zhang, Muhammad Adnan, Wafae Badi, Claudine Dereczynski, A Di Luca, Subimal Ghosh, Iskhaq Iskandar, James Kossin, Sophie Lewis, and 1 others. 2021. Weather and climate extreme events in a changing climate. find
  28. Cunshi Wang, Xinjie Hu, Yu Zhang, Xunhao Chen, Pengliang Du, Yiming Mao, Rui Wang, Yuyang Li, Ying Wu, Hang Yang, and 1 others. 2024a. Starwhisper telescope: Agent-based observation assistant system to approach ai astrophysicist. arXiv preprint arXiv:2412.06412. find
  29. Hao Wang, Jindong Han, Wei Fan, Leilei Sun, and Hao Liu. 2025a. Repst: language model empowered spatio-temporal forecasting via semantic-oriented reprogramming. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, pages 3362–3370. find
  30. Hao Wang, Jindong Han, Wei Fan, Weijia Zhang, and Hao Liu. 2025b. Phyda: Physics-guided diffusion models for data assimilation in atmospheric systems. arXiv preprint arXiv:2505.12882. — [arXiv:2505.12882](https://arxiv.org/abs/2505.12882) find
  31. Language Technologies (Volume 1: Long Papers), pages 11416–11431. find
  32. Shaolei Zhang, Ju Fan, Meihao Fan, Guoliang Li, and Xiaoyong Du. 2025. Deepanalyze: Agentic large language models for autonomous data science. arXiv preprint arXiv:2510.16872. — [arXiv:2510.16872](https://arxiv.org/abs/2510.16872) find
  33. Hao Wang, Zixuan Weng, Jindong Han, Wei Fan, and Hao Liu. 2025c. Dambench: A multi-modal benchmark for deep learning-based atmospheric data assimilation. arXiv preprint arXiv:2511.01468. — [arXiv:2511.01468](https://arxiv.org/abs/2511.01468) find
  34. Xiaoxuan Wang, Ziniu Hu, Pan Lu, Yanqiao Zhu, Jieyu Zhang, Satyen Subramaniam, Arjun R Loomba, Shichang Zhang, Yizhou Sun, and Wei Wang. 2024b. Scibench: Evaluating college-level scientific problem-solving abilities of large language models. In International Conference on Machine Learning, pages 506…
  35. Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, and 1 others. 2024c. Opendevin: An open platform for ai software developers as generalist agents. arXiv preprint arXiv:2407.16741, 3.
  36. Jiaqi Wei, Yuejin Yang, Xiang Zhang, Yuhan Chen, Xiang Zhuang, Zhangyang Gao, Dongzhan Zhou, Guangshuai Wang, Zhiqiang Gao, Juntai Cao, and 1 others. 2025. From ai for science to agentic science: A survey on autonomous scientific discovery. arXiv preprint arXiv:2508.14111. find
  37. Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. 2024a. Large language models for automated open-domain scientific hypotheses discovery. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13545–13565. find
  38. Zonglin Yang, Wanhao Liu, Ben Gao, Tong Xie, Yuqiang Li, Wanli Ouyang, Soujanya Poria, Erik Cambria, and Dongzhan Zhou. 2024b. Moose-chem: Large language models for rediscovering unseen chemistry scientific hypotheses. arXiv preprint arXiv:2410.07076. find
  39. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations.
  40. Mo Yu, Lemao Liu, Junjie Wu, Tsz Ting Chung, Shunchi Zhang, Jiangnan Li, Dit-Yan Yeung, and Jie Zhou. 2025. The stochastic parrot on llm’s shoulder: A summative assessment of physical concept understanding. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associati… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-climagent-llm-as-agents-autonomous/