Elamur Research⌕
☰
home / papers / 2025 / INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent

INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent

Paper · 2025 · Haohang Li, Yupeng Cao, Yangyang Yu et al.

Summary

LLM-based agents for finance are usually built for one narrow job, such as trading a single stock or only crypto, and are often tested on proprietary data. This makes them hard to compare. INVESTORBENCH addresses this gap as an open benchmark for agentic financial decision-making across three asset types: individual stocks, cryptocurrencies and ETFs. The authors extend the FINMEM agent into a general modular framework with backbone, perception, profile, memory and action components. Its memory has a working component and layered long-term storage, where each layer forgets at a different rate. Trading is formalised as a POMDP. The agent first goes through a warm-up phase with price feedback and then a test phase without future information, choosing Buy, Sell or Hold each day. The authors provide curated multimodal open datasets and environments and run 13 backbone LLMs, both proprietary and open-source. Results differ considerably by model and asset class: proprietary models generally trade stocks better, and open models struggle most in volatile crypto markets. The benchmark offers a shared, reproducible testbed for financial agents.

Key points

  • Covers three single-asset tasks: stock trading, cryptocurrency trading and ETF investing. Portfolio management is explicitly out of scope.
  • The agent builds on FINMEM. Its memory layers use distinct decay rates, which the authors favour over FINAGENT's purely similarity-based retrieval. Reflection is split into immediate and extended modes.
  • Thirteen LLMs serve as backbones, including GPT-4, GPT-4o, GPT-o1-preview, Llama-3.1-8B-Instruct, Qwen-2.5-Instruct-7B, Yi-1.5-9B-Chat and DeepSeek-V2-Lite.
  • The benchmark supports two usage modes:
    • plugging a fine-tuned LLM into the provided agent;
    • using only the environments and metrics with one's own agent design.
  • Code is available at github.com/felis33/INVESTOR-BENCH, and the data are released under the MIT license. Planned extensions include audio (earnings calls) and chart/graph modalities.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States241 works · 54,053 citations🎓Harvard UniversityCambridge · United States112 works · 42,023 citations🎓Stevens Institute of TechnologyHoboken · United States10 works · 95 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksDecision Sciences · 209 worksManagement Science and Operations Research · 108 works

🧭 Topics 1Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 works

Cite

Cite as Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.P. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow and Qianqian Xie. 2025. INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.126

@inproceedings{li2025investorbench,
  title = {{INVESTORBENCH: A Benchmark for Financial Decision-Making Tasks with LLM-based Agent}},
  author = {Haohang Li, Yupeng Cao, Yangyang Yu, Shashidhar Reddy Javaji, Zhiyang Deng, Yueru He, Yuechen Jiang, Zining Zhu, K.P. Subbalakshmi, Jimin Huang, Lingfei Qian, Xueqing Peng, Jordan W. Suchow and Qianqian Xie},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.126},
  url = {https://doi.org/10.18653/v1/2025.acl-long.126}
}

References

39 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Andrew Ang and Joseph Chen. 2003. Downside risk. Journal of Portfolio Management, 29(4):103–112. find
  3. Dogu Araci. 2019. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063. find
  4. Dimitri Bertsekas and Steven E Shreve. 1996. Stochastic optimal control: the discrete-time case, volume 5. Athena Scientific. find
  5. Mukul Bhatnagar, Sanjay Taneja, and Ramona Rupeika-Apoga. 2023. Demystifying the effect of the news (shocks) on crypto market volatility. Journal of Risk and Financial Management, 16(2):136. find
  6. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680.
  7. John Hull. 2007. Risk Management and Financial Institutions. John Wiley & Sons. find
  8. Taylan Kabbani and Ekrem Duman. 2022. Deep reinforcement learning approach for trading automation in the stock market. IEEE Access, 10:93564–93574. find
  9. Yuan Li, Bingqiao Luo, Qian Wang, Nuo Chen, Xu Liu, and Bingsheng He. 2024. Cryptotrade: A reflective llm-based agent to guide zero-shot cryptocurrency trading. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1094–1106. find
  10. Xiao-Yang Liu, Guoxuan Wang, and Daochen Zha. 2023. Fingpt: Democratizing internet-scale data for financial large language models. arXiv preprint arXiv:2307.10485. — [arXiv:2307.10485](https://arxiv.org/abs/2307.10485) find
  11. Yang Liu, Qi Liu, Hongke Zhao, Zhen Pan, and Chuanren Liu. 2020. Adaptive quantitative trading: An imitative deep reinforcement learning approach. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 2128–2135. find
  12. Zhuang Liu, Degen Huang, Kaiyu Huang, Zhuang Li, and Jun Zhao. 2021. Finbert: A pre-trained financial language representation model for financial text mining. In Proceedings of the twenty-ninth international conference on international joint conferences on artificial intelligence, pages 4513–4519. find
  13. Meta Llama team. 2024. The llama 3 herd of models.
  14. Ananth N Madhavan. 2016. Exchange-traded funds and the new dynamics of investing. Oxford University Press. find
  15. Jaap MJ Murre and Joeri Dros. 2015. Replication and analysis of ebbinghaus forgetting curve. PloS one, 10(7):e0120644. find
  16. OpenAI. 2022. Introducing chatgpt. find
  17. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, New York, NY, USA. Ass… find
  18. Alibaba Group Qwen team. 2024. Qwen2.5. find
  19. Raeid Saqur, Ken Kato, Nicholas Vinden, and Frank Rudzicz. 2024. Nifty financial news headlines dataset. arXiv preprint arXiv:2405.09747. — [arXiv:2405.09747](https://arxiv.org/abs/2405.09747) find
  20. Raj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, and Diyi Yang. 2022. When flue meets flang: Benchmarks and large pre-trained language model for financial domain. arXiv preprint arXiv:2211.00083. find
  21. William F. Sharpe. 1994. The sharpe ratio. The Journal of Portfolio Management, 21(1):49–58. find
  22. Writer Engineering team. 2024. Palmyra-Fin-70B-32k: a powerful LLM designed for Finance. https://dev.writer.com. ↗
  23. Olivier Vanhoucke. 2023. Crypto news dataset. Accessed: 2024-08-20. find
  24. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345.
  25. Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564.
  26. Qianqian Xie, Weiguang Han, Zhengyu Chen, Ruoyu Xiang, Xiao Zhang, Yueru He, Mengxi Xiao, Dong Li, Yongfu Dai, Duanyu Feng, et al. 2024. The finben: An holistic financial benchmark for large language models. arXiv preprint arXiv:2402.12659. — [arXiv:2402.12659](https://arxiv.org/abs/2402.12659) find
  27. Qianqian Xie, Weiguang Han, Xiao Zhang, Yanzhao Lai, Min Peng, Alejandro Lopez-Lira, and Jimin Huang. 2023. Pixiu: A large language model, instruction data and evaluation benchmark for finance. arXiv preprint arXiv:2306.05443. — [arXiv:2306.05443](https://arxiv.org/abs/2306.05443) find
  28. Huajian Xin, Z. Z. Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, Wenjun Gao, Qihao Zhu, Dejian Yang, Zhibin Gou, Z. F. Wu, Fuli Luo, and Chong Ruan. 2024. Deepseek-prover-v1.5: Harnessing proof assistant feedback for reinforcement learning and…
  29. Hongyang Yang, Boyu Zhang, Neng Wang, Cheng Guo, Xiaoli Zhang, Likun Lin, Junlin Wang, Tianyu Zhou, Mao Guan, Runjia Zhang, et al. 2024. Finrobot: An open-source ai agent platform for financial applications using large language models. arXiv preprint arXiv:2405.14767. — [arXiv:2405.14767](https://arxiv.org/abs/2405.14767) find
  30. Yi Yang, Yixuan Tang, and Kar Yan Tam. 2023. Investlm: A large language model for investment using financial domain instruction tuning. arXiv preprint arXiv:2309.13064. — [arXiv:2309.13064](https://arxiv.org/abs/2309.13064) find
  31. Yi Yang, Mark Christopher Siy Uy, and Allen Huang. 2020. Finbert: A pretrained language model for financial communications. arXiv preprint arXiv:2006.08097. find
  32. Ziruo Yi, Ting Xiao, Kaz-Onyeakazi Ijeoma, Ratnam Cheran, Yuvraj Baweja, and Phillip Nelson. 2022. Stock2vec: An embedding to improve predictive models for companies. arXiv preprint arXiv:2201.11290. — [arXiv:2201.11290](https://arxiv.org/abs/2201.11290) find
  33. Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al. 2024. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652.
  34. Yangyang Yu, Haohang Li, Zhi Chen, Yuechen Jiang, Yang Li, Denghui Zhang, Rong Liu, Jordan W Suchow, and Khaldoun Khashanah. 2024a. Finmem: A performance-enhanced llm trading agent with layered memory and character design. In Proceedings of the AAAI Symposium Series, volume 3, pages 595–597. find
  35. Yangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng, Yupeng Cao, Zhi Chen, Jordan W Suchow, Rong Liu, Zhenyu Cui, Zhaozhuo Xu, et al. 2024b. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making. arXiv preprint arXiv:2407.06567. — [arXiv:2407.06567](https://arxiv.org/abs/2407.06567) find
  36. Wentao Zhang, Lingxuan Zhao, Haochong Xia, Shuo Sun, Jiaze Sun, Molei Qin, Xinyi Li, Yuqing Zhao, Yilei Zhao, Xinyu Cai, et al. 2024a. Finagent: A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist. arXiv preprint arXiv:2402.18485. — [arXiv:2402.18485](https://arxiv.org/abs/2402.18485) find
  37. Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. 2024b. Llm as a mastermind: A survey of strategic reasoning with large language models. arXiv preprint arXiv:2404.01230. find
  38. Yuzhe Zhang and Hong Zhang. 2023. Finbert–mrc: financial named entity recognition using bert under the machine reading comprehension paradigm. Neural Processing Letters, 55(6):7393–7413. find
  39. Zhihan Zhou, Liqian Ma, and Han Liu. 2021. Trade the event: Corporate events detection for news-based event-driven trading. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2114–2124, Online. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2025-investorbench-benchmark-financial-decision-making/