Elamur Research⌕
☰
home / papers / 2023 / Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions

Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions

Paper · 2023 · Hui Yang, Sifu Yue and Yunzhong He

Summary

The paper asks how well Auto-GPT-style agents perform on online decision-making tasks that resemble real-world use. These agents receive only a high-level goal, reason in a self-monologue, call tools and keep memory. The authors adapt Auto-GPT to two interactive benchmarks, WebShop (simulated shopping over about 1.18M scraped Amazon products) and ALFWorld (text-based household tasks). The adaptation is minimal: the task goes in as the agent's goal, and each action is exposed as a tool with one to three few-shot demonstrations. They compare GPT-4, GPT-3.5, Claude and Vicuna against imitation-learning (IL) baselines. They also propose "Additional Opinions": the top-k actions suggested by an external expert model, here the existing IL policies, are inserted into the prompt as non-binding advice. This injects supervision without fine-tuning the LLM. GPT-4 is the strongest backbone and, with IL opinions, beats the task-specific IL models. The benefit depends on context, because weaker LLMs can be misled by unhelpful suggestions. The approach offers a cheap way to combine small expert models, such as recommenders or classifiers, with LLM agents.

Key points

  • WebShop setup: evaluation used the first 50 instructions to limit cost, with temperature 0.01. The baselines were a rule model, BART/BERT IL models with and without image features, and a random-suggestion ablation.
  • Main WebShop results: the best IL model reached 0.227 success and a reward of 57.689. Auto-GPT(GPT4) alone scored 0.240 success and 46.133 reward. Auto-GPT(GPT4) + IL(top5) reached 0.320 success and 61.550 reward. Auto-GPT(Vicuna) scored 0 on every metric.
  • Additional Opinions help across models on WebShop: GPT-3.5 success rose from 0.120 to 0.230, and Claude rose from 0.140 to 0.240. However, random suggestions dropped GPT-3.5 to 0.060.
  • ALFWorld: testing used 134 unseen games, with the DAgger-trained BUTLER::BRAIN text agent as baseline and expert. The IL advice there was often repetitive or misleading. GPT-4 and Claude mostly filtered it out, while GPT-3.5 was confused by it. Disagreement rates with the IL suggestions were 0.854 for GPT4, 0.814 for Claude and 0.769 for GPT3.5.
  • Code: github.com/younghuman/LLMAgent

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢AmazonSeattle · United States128 works · 41,697 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksEvaluation & benchmarks · 1,341 works

Cite

Cite as Hui Yang, Sifu Yue and Yunzhong He. 2023. Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions. arXiv:2306.02224.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2306.02224 · arXiv 2306.02224

@misc{yang2023auto,
  title = {{Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions}},
  author = {Hui Yang, Sifu Yue and Yunzhong He},
  year = {2023},
  doi = {10.48550/arxiv.2306.02224},
  eprint = {2306.02224},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.02224}
}

References

25 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, and Cameron McKinnon. 2022. Constitutional AI: Harmlessness from AI Feedback. In arXiv:2212.08073. arXiv. https://ar5iv.org/abs/2212.08073 ↗
  2. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of Artificial General Intelligence: Early experiments with GPT-4. (March 2023…
  3. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022).
  4. Marc-Alexandre Côté, Ákos Kádár, Xingdi Yuan, Ben Kybartas, Tavian Barnes, Emery Fine, James Moore, Matthew Hausknecht, Layla El Asri, Mahmoud Adada, et al. 2018. Textworld: A learning environment for text-based games. (2018), 41–75.
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
  6. Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2023. Complexity-Based Prompting for Multi-Step Reasoning. arXiv:2210.00720 [cs.CL]
  7. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019).
  8. Yang Li, Zhi-Ping Cai, and Hong Xu. 2018. LLMP: exploiting LLDP for latency measurement in software-defined data center networks. Journal of Computer Science and Technology 33 (2018), 277–285. find
  9. Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao, et al. 2023. Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis. arXiv preprint arXiv:2303.16434 (2023). — [arXiv:2303.16434](https://arxiv.org/abs/2303.16434) find
  10. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).
  11. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]
  12. Joon Sung Park, Joseph C. O’Brien, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. https://arxiv.org/pdf/2304.03442. pdf ↗
  13. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. Gorilla: Large Language Model Connected with Massive APIs. arXiv:2305.15334 [cs.CL]
  14. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2022. Instruction Tuning with GPT-4. https://ar5iv.org/abs/2304. 03277 ↗
  15. Stephane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. arXiv preprint arXiv:1011.0686 (2011). arXiv:1011.0686 find
  16. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace. arXiv:2303.17580 [cs.CL]
  17. Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366 (2023).
  18. Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks. arXiv preprint arXiv:1912.01734 (2020).
  19. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021. AlfWorld: Aligning Text and Embodied Environments for Interactive Learning. arXiv preprint arXiv:2010.03768 (2021).
  20. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. arXiv:2203.11171 [cs.CL]
  21. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022).
  22. Sherry Yang, Ofir Nachum, Yilun Du, Jason Wei, Pieter Abbeel, and Dale Schuurmans. 2023. Foundation Models for Decision Making: Problems, Methods, and Opportunities. arXiv e-prints (2023), arXiv–2303. find
  23. Ilan Yaniv. 2004. The benefit of additional opinions. Current directions in psychological science 13, 2 (2004), 75–78. find
  24. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. Webshop: Towards scalable real-world web interaction with grounded language agents. arXiv preprint arXiv:2207.01206 (2022).
  25. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2023-auto-gpt-online-decision-making/