AdaPlanner: Adaptive Planning from Feedback with Language Models

Paper · 2023 · Haotian Sun, Yuchen Zhuang, Lingkai Kong et al.

Summary

LLM agents for sequential decision-making usually either act step by step without any plan or follow a fixed plan, adjusting only the current action when the environment responds. Their performance drops as tasks grow longer and more complex. DEPS does revise whole plans, but it needs a trained plan selector and task-specific data. AdaPlanner is an explicit closed-loop method that works only through prompting. A single LLM acts as both planner and refiner. The planner splits a task into sub-goals and predicts what the environment should return. If an observation matches the prediction, the agent calls ask_LLM() to extract useful details and continues. If it contradicts the prediction, the plan is rewritten and execution resumes from the current point rather than restarting. Plans are written as code-style prompts to reduce hallucination. A skill memory stores successful plans and reuses them as few-shot examples. On ALFWorld and MiniWoB++, AdaPlanner beats state-of-the-art baselines while using far fewer demonstrations. This suggests that adaptive replanning makes training-free LLM agents both stronger and more sample-efficient.

Key points

AdaPlanner uses two refinement modes. In-plan refinement pulls information from expected observations via ask_LLM(). Out-of-plan refinement revises the whole plan when feedback deviates from the prediction.
It outperforms state-of-the-art baselines by 3.73% on ALFWorld (with 2x fewer samples) and by 4.11% on MiniWoB++ (with 600x fewer samples).
Experiments use GPT-3 and GPT-3.5, with Reflexion among the compared baselines. The ALFWorld evaluation covers 134 tasks.
In ablations, skill discovery nearly doubles the ALFWorld success rate and raises MiniWoB++ success by about 15%. Removing the code interface also lowers performance.
The authors note that complex tasks still require few-shot expert demonstrations. Code is released at github.com/haotiansun14/AdaPlanner.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields, topics and concepts

📚 Shelves 4Computer Science · 19,503 works Artificial Intelligence · 18,805 works Arts and Humanities · 17,959 works Language and Linguistics · 17,789 works

🧭 Topics 1Language models & pretraining · 4,145 works

Cite

Cite as Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai and Chao Zhang. 2023. AdaPlanner: Adaptive Planning from Feedback with Language Models. arXiv:2305.16653.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.16653 · arXiv 2305.16653

@misc{sun2023adaplanner,
  title = {{AdaPlanner: Adaptive Planning from Feedback with Language Models}},
  author = {Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai and Chao Zhang},
  year = {2023},
  doi = {10.48550/arxiv.2305.16653},
  eprint = {2305.16653},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.16653}
}

References

23 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
W. Chen, X. Ma, X. Wang, and W. W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv, page 2211.12588v3, 2022. find
L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig. Pal: Program-aided language models. arXiv, page 2211.10435v2, 2022. find
I. Gur, O. Nachum, Y. Miao, M. Safdari, A. Huang, A. Chowdhery, S. Narang, N. Fiedel, and A. Faust. Understanding html with large language models. arXiv, page 2210.03945v1, 2022. find
W. Huang, P. Abbeel, D. Pathak, and I. Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. arXiv, page 2201.07207v2, 2022.
W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, P. Sermanet, N. Brown, T. Jackson, L. Luu, S. Levine, K. Hausman, and B. Ichter. Inner monologue: Embodied reasoning through planning with language models. arXiv, page 2207.05608v1, 2022. find
P. C. Humphreys, D. Raposo, T. Pohlen, G. Thornton, R. Chhaparia, A. Muldal, J. Abramson, P. Georgiev, A. Goldin, A. Santoro, and T. Lillicrap. A data-driven approach for learning to control computers. arXivProceedings of the 39th International Conference on Machine Learning, Baltimore, Maryland, US… find
E. Jang. Can llms critique and iterate on their own outputs? evjang.com, 2023. find
G. Kim, P. Baldi, and S. McAleer. Language models can solve computer tasks. arXiv, page 2303.17491v1, 2023.
J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng. Code as policies: Language model programs for embodied control. arXiv, page 2209.07753v3, 2022. find
E. Z. Liu, K. Guu, P. Pasupat, and P. Liang. Reinforcement learning on web interfaces using workflow-guided exploration. In International Conference on Learning Representations, 2018.
P. Lu, B. Peng, H. Cheng, M. Galley, K.-W. Chang, Y. N. Wu, S.-C. Zhu, and J. Gao. Chameleon: Plug-and-play compositional reasoning with large language models. arXiv preprint arXiv:2304.09842, 2023.
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
A. Parisi, Y. Zhao, and N. Fiedel. Talm: Tool augmented language models. arXiv preprint arXiv:2205.12255, 2022.
T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023.
Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580, 2023.
N. Shinn, B. Labash, and A. Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366, 2023.
M. Shridhar, X. Yuan, M.-A. Cote, Y. Bisk, A. Trischler, and M. Hausknecht. {ALFW}orld: Aligning text and embodied environments for interactive learning. In International Conference on Learning Representations, 2021. find
I. Singh, V. Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg. Progprompt: Generating situated robot task plans using large language models. In Second Workshop on Language and Reinforcement Learning, 2022.
Z. Wang, S. Cai, A. Liu, X. Ma, and Y. Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv, page 2302.01560v1, 2023.
J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou. Chain-of-thought prompting elicits reasoning in large language models. arXiv, page 2201.11903v6, 2022.
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023.
D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le, and E. Chi. Least-to-most prompting enables complex reasoning in large language models. arXiv, page 2205.10625v2, 2022.

Updated 2 November 2026

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sun-2023-adaplanner-adaptive-planning-feedback-language/