Elamur Research⌕
☰
home / papers / 2024 / QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model

QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model

Paper · 2024 · Saizhuo Wang, Hang Yuan, Lionel M. Ni et al.

Summary

Adapting LLM agents to specialised fields such as quantitative investment is difficult because a high-quality domain knowledge base (for example, a library of financial alphas) is expensive or impractical to assemble by hand. The authors (HKUST and IDEA Research) propose a self-improving framework with two nested loops. In the inner loop, a "writer" LLM drafts answers and a "judge" LLM scores and critiques them, with both drawing hints from the knowledge base through a shared context buffer. In the outer loop, the final answers are evaluated in the real environment, and the resulting feedback is written back into the knowledge base. Framing this process as a Markov Decision Process, they argue that both loops converge efficiently toward optimal behaviour under certain assumptions. They implement the framework as QuantAgent, which mines trading signals as code from LLM-generated trading ideas. In backtests on Chinese stocks, QuantAgent's signals improve in quality and relevance over iterations, and models trained on the accumulated signals forecast more accurately. This shows a route to building a domain knowledge base with little human labour.

Key points

  • Two-loop architecture:
    • The inner loop (writer, judge, knowledge retrieval and context buffer) stops when the score reaches a threshold β or after a maximum of T iterations.
    • The outer loop runs K rounds of real-world feedback that update the knowledge base.
  • Knowledge base contents: Each stored signal includes its code, the underlying trading idea, performance metrics and reviews.
  • Signal generation:
    • Each signal is a function written into a predefined template, an approach similar to FunSearch.
    • Trading ideas are sampled from another LLM.
  • Data: 500 stocks from the Chinese A-share market, using 2023 daily price and volume data.
  • Evaluation:
    • Single-signal metrics: entry valid ratio, IC, return and Sharpe ratio.
    • GPT-4 judged how faithfully each signal reflects its trading idea; ablations removed the inner loop, the outer loop, or both.
    • The MSE of the forecasting model fell as more signals accumulated.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Hong Kong University of Science and TechnologyHong Kong · China183 works · 27,600 citations🎓The Hong Kong University of Science and Technology (Guangzhou)Guangzhou · China73 works · 1,936 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksDecision Sciences · 306 worksManagement Science and Operations Research · 145 worksEconomics, Econometrics and Finance · 24 worksFinance · 7 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Saizhuo Wang, Hang Yuan, Lionel M. Ni and Jian Guo. 2024. QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model. arXiv:2402.03755.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2402.03755 · arXiv 2402.03755

@misc{wang2024quantagent,
  title = {{QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model}},
  author = {Saizhuo Wang, Hang Yuan, Lionel M. Ni and Jian Guo},
  year = {2024},
  doi = {10.48550/arxiv.2402.03755},
  eprint = {2402.03755},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.03755}
}

References

22 references, 11 of them with a card of ours. ↗ leads to the work.

  1. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Guo, Q., Wang, M., and Wang, H. Retrieval-Augmented Generation for Large Language Models: A Survey, January 2024. URL http://arxiv.org/ abs/2312.10997. arXiv:2312.10997 [cs]. ↗
  2. Haluptzok, P., Bowers, M., and Kalai, A. T. Language Models Can Teach Themselves to Program Better. September 2022. URL https://openreview.net/forum? id=SaRj2ka1XZ3. ↗
  3. Schick, T., Dwivedi-Yu, J., Dessi, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and Scialom, T. Toolformer: Language Models Can Teach Themselves to Use Tools. November 2023. URL https://openreview.net/forum?id=Yacmpz84TH. ↗
  4. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. LoRA: Low-Rank Adaptation of Large Language Models, October 2021. URL http://arxiv.org/abs/2106. 09685. arXiv:2106.09685 [cs]. ↗
  5. Significant Gravitas. AutoGPT, February 2024. URL https://github.com/ Significant-Gravitas/AutoGPT. original-date: 2023-03-16T09:21:07Z. ↗
  6. Jin, Y., Yang, Z., and Wang, Z. Is Pessimism Provably Efficient for Offline RL? In Proceedings of the 38th International Conference on Machine Learning, pp. 5084–5096. PMLR, July 2021. URL https://proceedings.mlr.press/v139/jin21e.html. ISSN: 2640-3498. ↗
  7. Sumers, T. R., Yao, S., Narasimhan, K., and Griffiths, T. L. Cognitive Architectures for Language Agents, September 2023. URL http://arxiv.org/abs/2309. 02427. arXiv:2309.02427 [cs]. ↗
  8. Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense Passage Retrieval for Open-Domain Question Answering, September 2020. URL http://arxiv.org/abs/2004. 04906. arXiv:2004.04906 [cs]. ↗
  9. Sun, J., Xu, C., Tang, L., Wang, S., Lin, C., Gong, Y., Ni, L., Shum, H.-Y., and Guo, J. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph. October 2023. URL https://openreview.net/forum?id=nnVO1PvbTv. ↗
  10. Liu, Z., Hu, H., Zhang, S., Guo, H., Ke, S., Liu, B., and Wang, Z. Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency, October 2023. URL http://arxiv.org/abs/2309.17382. arXiv:2309.17382 [cs]. ↗
  11. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hossei… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  12. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K. R. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. November 2023. URL https://openreview.net/forum?id=5Xc1ecxO1h. ↗
  13. Trinh, T. H., Wu, Y., Le, Q. V., He, H., and Luong, T. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476–482, January 2024. ISSN 1476-4687. doi: 10.1038/s41586-023-06747-5. URL https://doi.org/10.1038/s41586-023-06747-5. ↗
  14. Zhu, Z., Xue, Y., Chen, X., Zhou, D., Tang, J., Schuurmans, D., and Dai, H. Large Language Models can Learn Rules, October 2023. URL http://arxiv.org/abs/2310.07064. arXiv:2310.07064 [cs]. — [arXiv:2310.07064](https://arxiv.org/abs/2310.07064) ↗
  15. Tulchinsky, I. Introduction to Alpha Design. In Finding Alphas, pp. 1–6. John Wiley & Sons, Ltd, 2019. ISBN 978-1-119-57127-8. doi: https://doi.org/10.1002/9781119571278.ch1. URL https://onlinelibrary.wiley.com/doi/ abs/10.1002/9781119571278.ch1. — [doi:10.1002/9781119571278.ch1](https://doi.org/10.1002/9781119571278.ch1) ↗
  16. Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A. Voyager: An Open-Ended Embodied Agent with Large Language Models, October 2023a. URL http://arxiv.org/abs/ 2305.16291. arXiv:2305.16291 [cs]. ↗
  17. Wang, S., Liu, Z., Wang, Z., and Guo, J. A Principled Framework for Knowledge-enhanced Large Language Model, November 2023b. URL http://arxiv.org/ abs/2311.11135. arXiv:2311.11135 [cs]. — [arXiv:2311.11135](https://arxiv.org/abs/2311.11135) ↗
  18. Wang, S., Yuan, H., Zhou, L., Ni, L. M., Shum, H.-Y., and Guo, J. Alpha-GPT: Human-AI Interactive Alpha Mining for Quantitative Investment, July 2023c. URL http://arxiv.org/abs/2308.00016. arXiv:2308.00016 [cs, q-fin]. — [arXiv:2308.00016](https://arxiv.org/abs/2308.00016) ↗
  19. Wang, Y., Zhong, W., Li, L., Mi, F., Zeng, X., Huang, W., Shang, L., Jiang, X., and Liu, Q. Aligning Large Language Models with Human: A Survey, July 2023d. URL http://arxiv.org/abs/2307. 12966. arXiv:2307.12966 [cs]. ↗
  20. Weng, L. LLM Powered Autonomous Agents, June 2023. URL https://lilianweng.github.io/ posts/2023-06-23-agent/. Section: posts. ↗
  21. Xie, S. M., Raghunathan, A., Liang, P., and Ma, T. An Explanation of In-context Learning as Implicit Bayesian Inference. October 2021. URL https://openreview.net/forum?id=RdJVFCHjUMI. ↗
  22. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. ReAct: Synergizing Reasoning and Acting in Language Models. September 2022. URL https://openreview.net/forum? id=WE_vluYUL-X. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2024-quantagent-seeking-holy-grail-trading/