Reflexion: Language Agents with Verbal Reinforcement Learning

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Computing & GPUs

Cited by

1272 papers of our library cite it (of the 140673 whose bibliographies we read); 52 of them have a page here, 1220 not yet.

• Quiet-STaR: Language models can teach themselves to think before speaking
• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors
• Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
• Improving Factuality and Reasoning in Language Models through Multiagent Debate
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
• CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
• Large Language Model based Multi-Agents: A Survey of Progress and Challenges
• PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering
• Token-Budget-Aware LLM Reasoning
• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
• Large Language Models Cannot Self-Correct Reasoning Yet
• MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• Survey of Hallucination in Natural Language Generation
• A Survey on Large Language Models for Code Generation
• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
• Language Models can Solve Computer Tasks
• Training Language Models to Self-Correct via Reinforcement Learning
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
• LegalAgentBench: Evaluating LLM Agents in Legal Domain
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
• Deductive Verification of Chain-of-Thought Reasoning
• AgentBench: Evaluating LLMs as Agents
• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
• A Comprehensive Overview of Large Language Models
• AlphaEvolve: A coding agent for scientific and algorithmic discovery
• Gorilla: Large Language Model Connected with Massive APIs
• REFINER: Reasoning Feedback on Intermediate Representations
• Reasoning with Language Model Prompting: A Survey
• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
• UI-TARS: Pioneering Automated GUI Interaction with Native Agents
• Agent Laboratory: Using LLM Agents as Research Assistants
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
• Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
• Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents
• MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback
• Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
• Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration
• OpenHands: An Open Platform for AI Software Developers as Generalist Agents
• The Rise and Potential of Large Language Model Based Agents: A Survey
• Self-Evaluation Guided Beam Search for Reasoning
• The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
• The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)
• Large Language Models as Optimizers
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
• DAPO: An Open-Source LLM Reinforcement Learning System at Scale
• A Survey of Large Language Models
• WebArena: A Realistic Web Environment for Building Autonomous Agents
• ToolQA: A Dataset for LLM Question Answering with External Tools
• BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.11366 · arXiv 2303.11366

@misc{shinn2023reflexion,
  title = {{Reflexion: Language Agents with Verbal Reinforcement Learning}},
  author = {Shinn, Noah and Cassano, Federico and Berman, Edward and Gopinath, Ashwin and Narasimhan, Karthik and Yao, Shunyu},
  year = {2023},
  doi = {10.48550/arxiv.2303.11366},
  eprint = {2303.11366},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.11366}
}

References

31 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., et al. (2022). Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691. find our card
2. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021). Program synthesis with large language models. arXiv preprint arXiv:2108.07732. find our card
3. Brooks, E., Walls, L., Lewis, R. L., and Singh, S. (2022). In-context policy iteration. arXiv preprint arXiv:2210.03821. — [arXiv:2210.03821](https://arxiv.org/abs/2210.03821) find
4. Cassano, F., Gouwar, J., Nguyen, D., Nguyen, S., Phipps-Costin, L., Pinckney, D., Yee, M.-H., Zi, Y., Anderson, C. J., Feldman, M. Q., Guha, A., Greenberg, M., and Jangda, A. (2022). Multipl-e: A scalable and extensible approach to benchmarking neural code generation. find
5. Chen, B., Zhang, F., Nguyen, A., Zan, D., Lin, Z., Lou, J.-G., and Chen, W. (2022). Codet: Code generation with generated tests. arXiv preprint arXiv:2207.10397. find
6. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. find our card
7. Chen, X., Lin, M., Schärli, N., and Zhou, D. (2023). Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128. find
8. Côté, M.-A., Kádár, A., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J., Hausknecht, M., El Asri, L., Adada, M., et al. (2019). Textworld: A learning environment for text-based games. In Computer Games: 7th Workshop, CGW 2018, Held in Conjunction with the 27th International Conference on Art… find our card
9. Goodman, N. (2023). Meta-prompt: A simple self-improving language agent. noahgood-man.substack.com. find
10. Kim, G., Baldi, P., and McAleer, S. (2023). Language models can solve computer tasks. arXiv preprint arXiv:2303.17491. find our card
11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). find
12. Le, H., Wang, Y., Gotmare, A. D., Savarese, S., and Hoi, S. C. H. (2022). Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 35:21314–21328. find our card
13. Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al. (2023). Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161. find
14. Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022). Competition-level code generation with alphacode. Science, 378(6624):1092–1097. find our card
15. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023). Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. find
16. Nair, V., Schumacher, E., Tso, G., and Kannan, A. (2023). Dera: Enhancing large language model completions with dialog-enabled resolving agents. arXiv preprint arXiv:2303.17071. find
17. Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021). Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332. find our card
18. OpenAI (2023). Gpt-4 technical report. ArXiv. find our card
19. Park, J. S., O’Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. find
20. Paul, D., Ismayilzada, M., Peyrard, M., Borges, B., Bosselut, A., West, R., and Faltings, B. (2023). Refiner: Reasoning feedback on intermediate representations. arXiv preprint arXiv:2304.01904. find our card
21. Pryzant, R., Iter, D., Li, J., Lee, Y. T., Zhu, C., and Zeng, M. (2023). Automatic prompt optimization with" gradient descent" and beam search. arXiv preprint arXiv:2305.03495. find our card
22. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., and Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761. find our card
23. Shen, Y., Song, K., Tan, X., Li, D., Lu, W., and Zhuang, Y. (2023). Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580. find our card
24. Shridhar, M., Yuan, X., Côté, M.-A., Bisk, Y., Trischler, A., and Hausknecht, M. (2021). ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. In Proceedings of the International Conference on Learning Representations (ICLR). find our card
25. Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. The MIT Press, second edition. find
26. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D. (2022). Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. find our card
27. Xie, Y., Kawaguchi, K., Zhao, Y., Zhao, X., Kan, M.-Y., He, J., and Xie, Q. (2023). Decomposition enhances reasoning via self-evaluation guided decoding. arXiv preprint arXiv:2305.00633. find our card
28. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. (2018). HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP). find our card
29. Yao, S., Chen, H., Yang, J., and Narasimhan, K. (preprint). Webshop: Towards scalable real-world web interaction with grounded language agents. In ArXiv. find our card
30. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR). find our card
31. Yoran, O., Wolfson, T., Bogin, B., Katz, U., Deutch, D., and Berant, J. (2023). Answering questions by meta-reasoning over multiple chains of thought. arXiv preprint arXiv:2304.13007. find