Elamur Research
☰
home / papers / 2023 / FinGPT: Open-Source Financial Large Language Models

FinGPT: Open-Source Financial Large Language Models

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2306.06031

Summary

Financial text is hard for LLMs because it is time-critical, changes constantly, and contains much more noise than useful signal. Proprietary systems like BloombergGPT rely on exclusive data and opaque training. This paper introduces FinGPT, an open-source, data-centric framework for building custom financial LLMs. It has five layers:

  • a data source layer covering news, social media, filings and trend sites
  • a real-time data engineering layer
  • an LLM layer built on lightweight fine-tuning
  • a tasks layer that also serves as a benchmark
  • an applications layer, for example robo-advising

The authors test the framework on financial news sentiment classification. They first fine-tune Llama-3.1-8B-Instruct with LoRA. They then apply Reinforcement Learning on Stock Prices (RLSP), where the reward comes from how the stock price moves after a news item. Training labels are produced automatically from short-term price changes, so no manual annotation is needed. The resulting model beats FinBERT and zero-shot ChatGPT and Llama3.1-8B. The work matters because it offers a cheap, transparent alternative to closed FinLLMs. The code is released in the FinGPT and FinNLP repositories of the AI4Finance Foundation.

Key points

  • Dataset: more than 620,000 cleaned headlines from 2016–2024, collected through the FinNLP pipeline from sources such as CNBC, Reuters, Yahoo Finance and MarketWatch. Each headline is labeled Positive, Negative or Neutral by comparing the price change after the news against thresholds.

  • LoRA setup: rank r = 8, α = 16, about 8.3M trainable parameters (under 0.1% of the 8B model), batch size 64, learning rate 2×10⁻⁴, 3 epochs.

  • Results (Table 1):

    ModelAccuracyMacro-F1
    FinGPT (SFT+RLSP)82.180.9
    FinGPT (LoRA-SFT only)78.877.3
    FinBERT71.269.9
    ChatGPT (zero-shot)63.461.7
    Llama3.1-8B (zero-shot)57.954.4

  • Ablation: Macro-F1 rises from 54.4 (base model) to 77.3 with LoRA and to 80.9 with RLSP added. Most of the gain comes from LoRA, while RLSP improves agreement with actual market reactions.

  • Caveat: BloombergGPT is named as a baseline, but no scores for it appear in the results table.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

126 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 122 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2306.06031 · arXiv 2306.06031

@misc{yang2023fingpt,
  title = {{FinGPT: Open-Source Financial Large Language Models}},
  author = {Yang, Hongyang and Liu, Xiao-Yang and Wang, Christina Dan},
  year = {2023},
  doi = {10.48550/arxiv.2306.06031},
  eprint = {2306.06031},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.06031}
}

References

28 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dogu Araci. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063, 2019. — [arXiv:1908.10063](https://arxiv.org/abs/1908.10063) find
  2. Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhihua Wu, Zhen Guo, Hua Lu, Xinxian Huang, et al. Plato-xl: Exploring the large-scale pre-training of dialogue generation. arXiv preprint arXiv:2109.09519, 2021. find
  3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877–1901, 2020. find
  4. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, et al. Finqa: A dataset of numerical reasoning over financial data. arXiv preprint arXiv:2109.00122, 2021. find our card
  5. Alexandra DeLucia, Shijie Wu, Aaron Mueller, Carlos Aguirre, Philip Resnik, and Mark Dredze. Bernice: a multilingual pre-trained encoder for Twitter. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 6191–6205, 2022. find
  6. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. arXiv preprint arXiv:2305.14314, 2023. find
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. find
  8. Mark Dredze, Prabhanjan Kambadur, Gary Kazantsev, Gideon Mann, and Miles Osborne. How twitter is changing the nature of financial news discovery. In Proceedings of the second International Workshop on Data Science for Macro-modeling, pages 1–5, 2016. find
  9. Kawin Ethayarajh. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. arXiv preprint arXiv:1909.00512, 2019. find
  10. Sohom Ghosh, Shovon Sengupta, Sudip Naskar, and Sunny Kumar Singh. FinRead: A transfer learning based tool to assess readability of definitions of financial terms. In Proceedings of the 18th International Conference on Natural Language Processing (ICON), pages 658–659, National Institute of Technolo… find
  11. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. find
  12. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations, 2021. find
  13. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroi… find
  14. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019. find our card
  15. Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. Pretrained language models for biomedical and clinical tasks: understanding and extending the state-of-the-art. In Proceedings of the 3rd Clinical Natural Language Processing Workshop, pages 146–157, 2020. find
  16. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. find
  17. OpenAI. Chatgpt. https://chat.openai.com/, 2023. Large language model accessed via ChatGPT interface. ↗
  18. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022. find
  19. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. OpenAI, 2018. find
  20. Raj Sanjay Shah, Kunal Chawla, Dheeraj Eidnani, Agam Shah, Wendi Du, Sudheer Chava, Natraj Raman, Charese Smiley, Jiaao Chen, and Diyi Yang. When flue meets flang: Benchmarks and large pre-trained language model for financial domain. arXiv preprint arXiv:2211.00083, 2022. — [arXiv:2211.00083](https://arxiv.org/abs/2211.00083) find
  21. Soumya Sharma, Tapas Nayak, Arusarka Bose, Ajay Kumar Meena, Koustuv Dasgupta, Niloy Ganguly, and Pawan Goyal. Finred: A dataset for relation extraction in financial domain. In Companion Proceedings of the Web Conference 2022, WWW ’22, page 595–597, New York, NY, USA, 2022. Association for Computing… find
  22. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. find our card
  23. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022. find our card
  24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. find
  25. Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. BloombergGPT: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023. find our card
  26. Hongyang Yang, Xiao-Yang Liu, Shan Zhong, and Anwar Walid. Deep reinforcement learning for automated stock trading: An ensemble strategy. In Proceedings of the first ACM international conference on AI in finance, pages 1–8, 2020. find
  27. Hongyang Yang, Boyu Zhang, Neng Wang, Cheng Guo, Xiaoli Zhang, Likun Lin, Junlin Wang, Tianyu Zhou, Mao Guan, Runjia Zhang, et al. Finrobot: An open-source ai agent platform for financial applications using large language models. arXiv preprint arXiv:2405.14767, 2024. — [arXiv:2405.14767](https://arxiv.org/abs/2405.14767) find
  28. Boyu Zhang, Hongyang Yang, Tianyu Zhou, Ali Babar, and Xiao-Yang Liu. Enhancing financial sentiment analysis via retrieval augmented large language models. ACM International Conference on AI in Finance (ICAIF), 2023. find