Elamur Research⌕
☰
home / papers / 2023 / Generalized Planning in PDDL Domains with Pretrained Large Language Models

Generalized Planning in PDDL Domains with Pretrained Large Language Models

Paper · 2023 · Tom Silver, Soham Dan, Kavitha Srinivas et al.

Summary

The paper asks whether a pretrained LLM can be a generalized planner rather than a per-task planner. Given a PDDL domain and a few training tasks, the model should produce a program that quickly outputs valid plans for new, usually much larger tasks in that domain. GPT-4 receives the domain and two abbreviated training tasks, then writes a Python plan generator. The prompt explicitly steers it away from search-based code. Two add-ons are studied. The first is a Chain-of-Thought stage, where the model describes the domain and a solution strategy in plain language before coding. The second is an automatic repair loop that tests the code on training tasks and returns error feedback for up to four rounds. The method is evaluated on seven domains against ablations and baselines, including PG3. GPT-4 proves a surprisingly capable generalized planner, though PG3 still performs better. This matters because synthesized programs can be inspected, are cheap to run, and are not bounded by the context window, unlike querying an LLM for every task.

Key points

  • Evaluation uses seven PDDL domains: six from the PG3 work (Yang et al. 2022) and one new domain. The method is compared with four ablations and four baselines. A task counts as solved if a valid plan appears within 30 seconds.
  • Automated debugging gives four feedback types: Python exceptions, semantic plan errors, plan syntax errors, and timeouts. Even one repair round helps a lot, and later rounds add less. Chain-of-Thought summarization has mixed effects across domains.
  • Replacing human-readable PDDL names (the No Names ablation) severely hurts results. GPT-3.5 performs far worse than GPT-4 and gains little from debugging.
  • At most 10 training tasks are available, but only two appear in the prompt, with object and atom lists truncated beyond 10 entries. Successful runs rarely need more than those two tasks.
  • Code and logs are released at github.com/tomsilver/llm-genplan. The authors argue classical planning stays necessary where no simple program exists, such as Sokoban or Slitherlink.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢IBMArmonk · United States163 works · 73,139 citations🔬MIT Computer Science and Artificial Intelligence LaboratoryCambridge · United States80 works · 32,402 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works

Cite

Cite as Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B. Tenenbaum, Leslie Pack Kaelbling and Michael Katz. 2023. Generalized Planning in PDDL Domains with Pretrained Large Language Models. arXiv:2305.11014.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.11014 · arXiv 2305.11014

@misc{silver2023generalized,
  title = {{Generalized Planning in PDDL Domains with Pretrained Large Language Models}},
  author = {Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B. Tenenbaum, Leslie Pack Kaelbling and Michael Katz},
  year = {2023},
  doi = {10.48550/arxiv.2305.11014},
  eprint = {2305.11014},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.11014}
}

References

53 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahn, M.; Brohan, A.; Brown, N.; Chebotar, Y.; Cortes, O.; David, B.; Finn, C.; Gopalakrishnan, K.; Hausman, K.; Herzog, A.; et al. 2022. Do as I can, not as I say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691.
  2. Alur, R.; Bodik, R.; Juniwal, G.; Martin, M. M.; Raghothaman, M.; Seshia, S. A.; Singh, R.; Solar-Lezama, A.; Torlak, E.; and Udupa, A. 2013. Syntax-guided synthesis. IEEE.
  3. Bonet, B.; and Geffner, H. 2015. Policies that generalize: Solving many planning problems with the same policy. In Twenty-Fourth International Joint Conference on Artificial Intelligence. find
  4. Bonet, B.; and Geffner, H. 2018. Features, Projections, and Representation Change for Generalized Planning. CoRR, abs/1801.10055. — [arXiv:1801.10055](https://arxiv.org/abs/1801.10055) find
  5. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877– 1901.
  6. Bubeck, S.; Chandrasekaran, V.; Eldan, R.; Gehrke, J.; Horvitz, E.; Kamar, E.; Lee, P.; Lee, Y. T.; Li, Y.; Lundberg, S.; et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712.
  7. Chapman, D. 1987. Planning for Conjunctive Goals. Artificial Intelligence, 32: 333–377. find
  8. Chen, A.; Scheurer, J.; Korbak, T.; Campos, J. A.; Chan, J. S.; Bowman, S. R.; Cho, K.; and Perez, E. 2023a. Improving Code Generation by Training with Natural Language Feedback. arXiv:2303.16749. find
  9. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; Pinto, H. P. d. O.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  10. Chen, X.; Lin, M.; Schärli, N.; and Zhou, D. 2023b. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128. find
  11. Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.
  12. Collins, K. M.; Wong, C.; Feng, J.; Wei, M.; and Tenenbaum, J. B. 2022. Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks. arXiv preprint arXiv:2205.05718. — [arXiv:2205.05718](https://arxiv.org/abs/2205.05718) find
  13. Cropper, A.; and Dumančić, S. 2022. Inductive logic programming at 30: a new introduction. Journal of Artificial Intelligence Research, 74: 765–850. find
  14. Fikes, R. E.; Hart, P. E.; and Nilsson, N. J. 1972. Learning and executing generalized robot plans. Artificial intelligence, 3: 251–288.
  15. Gao, L.; Madaan, A.; Zhou, S.; Alon, U.; Liu, P.; Yang, Y.; Callan, J.; and Neubig, G. 2022. PAL: Program-aided Language Models. arXiv preprint arXiv:2211.10435. find
  16. Gulwani, S.; Polozov, O.; Singh, R.; et al. 2017. Program synthesis. Foundations and Trends® in Programming Languages, 4(1-2): 1–119.
  17. Helmert, M. 2006. The fast downward planning system. Journal of Artificial Intelligence Research, 26: 191–246.
  18. Howey, R.; Long, D.; and Fox, M. 2004. VAL: Automatic plan validation, continuous effects and mixed initiative planning using PDDL. In 16th IEEE International Conference on Tools with Artificial Intelligence, 294–301. find
  19. IEEE. Huang, W.; Abbeel, P.; Pathak, D.; and Mordatch, I. 2022a. Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents. In International Conference on Machine Learning (ICML).
  20. Huang, W.; Xia, F.; Xiao, T.; Chan, H.; Liang, J.; Florence, P.; Zeng, A.; Tompson, J.; Mordatch, I.; Chebotar, Y.; et al. 2022b. Inner Monologue: Embodied Reasoning through Planning with Language Models. arXiv preprint arXiv:2207.05608. find
  21. Imani, S.; Du, L.; and Shrivastava, H. 2023. MathPrompter: Mathematical Reasoning using Large Language Models. arXiv:2303.05398.
  22. Jiang, X.; Dong, Y.; Wang, L.; Shang, Q.; and Li, G. 2023. Self-planning Code Generation with Large Language Model. arXiv preprint arXiv:2303.06689. Jiménez, S.; and Jonsson, A. 2015. Computing plans with control flow and procedures using a classical planner. In Proceedings of the Eighth Annual Sym…
  23. Katz, M.; and Sohrabi, S. 2020. Reshaping diverse planning. In Proceedings of the AAAI Conference on Artificial Intelligence, 06, 9892–9899. find
  24. Levesque, H. 2005. Planning with Loops. In IJCAI. find
  25. Levine, J.; and Humphreys, D. 2003. Learning action strategies for planning domains using genetic programming. In Workshops on Applications of Evolutionary Computation, 684–695. find
  26. Springer. Liang, J.; Huang, W.; Xia, F.; Xu, P.; Hausman, K.; Ichter, B.; Florence, P.; and Zeng, A. 2022. Code as policies: Language model programs for embodied control. arXiv preprint arXiv:2209.07753. find
  27. Lin, B. Y.; Huang, C.; Liu, Q.; Gu, W.; Sommerer, S.; and Ren, X. 2022. On Grounded Planning for Embodied Tasks with Language Models. arXiv preprint arXiv:2209.00465. — [arXiv:2209.00465](https://arxiv.org/abs/2209.00465) find
  28. Lin, K.; Agia, C.; Migimatsu, T.; Pavone, M.; and Bohg, J. 2023. Text2motion: From natural language instructions to feasible plans. arXiv preprint arXiv:2303.12153. — [arXiv:2303.12153](https://arxiv.org/abs/2303.12153) find
  29. Liu, B.; Jiang, Y.; Zhang, X.; Liu, Q.; Zhang, S.; Biswas, J.; and Stone, P. 2023. LLM+ P: Empowering Large Language Models with Optimal Planning Proficiency. arXiv preprint arXiv:2304.11477.
  30. McDermott, D. 2000. The 1998 AI Planning Systems Competition. AI Magazine, 21(2): 35–55. find
  31. Muggleton, S. 1991. Inductive logic programming. New generation computing, 8: 295–318. find
  32. Nijkamp, E.; Pang, B.; Hayashi, H.; Tu, L.; Wang, H.; Zhou, Y.; Savarese, S.; and Xiong, C. 2023. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. arXiv:2203.13474. find
  33. OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774.
  34. Pallagani, V.; Muppasani, B.; Murugesan, K.; Rossi, F.; Horesh, L.; Srivastava, B.; Fabiano, F.; and Loreggia, A. 2022. Plansformer: Generating Symbolic Plans using Transformers. arXiv preprint arXiv:2212.08681. — [arXiv:2212.08681](https://arxiv.org/abs/2212.08681) find
  35. Raman, S. S.; Cohen, V.; Rosen, E.; Idrees, I.; Paulius, D.; and Tellex, S. 2022. Planning with Large Language Models via Corrective Re-prompting. arXiv preprint arXiv:2211.09935. — [arXiv:2211.09935](https://arxiv.org/abs/2211.09935) find
  36. Richter, S.; and Westphal, M. 2010. The LAMA planner: Guiding Cost-based Anytime Planning with Landmarks. Journal of Artificial Intelligence Research, 39: 127–177. find
  37. Rivlin, O.; Hazan, T.; and Karpas, E. 2020. Generalized Planning With Deep Reinforcement Learning. arXiv preprint arXiv:2005.02305. — [arXiv:2005.02305](https://arxiv.org/abs/2005.02305) find
  38. Schaul, T.; Horgan, D.; Gregor, K.; and Silver, D. 2015. Universal value function approximators. In International conference on machine learning, 1312–1320. PMLR. Schick, T.; Dwivedi-Yu, J.; Dessı̀, R.; Raileanu, R.; Lomeli, M.; Zettlemoyer, L.; Cancedda, N.; and Scialom, T. 2023. Toolformer: Langua…
  39. Sharma, P.; Torralba, A.; and Andreas, J. 2022. Skill Induction and Planning with Latent Language. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1713–1726.
  40. Silver, T.; Chitnis, R.; Kumar, N.; McClinton, W.; Lozano-Perez, T.; Kaelbling, L. P.; and Tenenbaum, J. 2023. Predicate Invention for Bilevel Planning. In AAAI Conference on Artificial Intelligence (AAAI). Silver, T.; Hariprasad, V.; Shuttleworth, R. S.; Kumar, N.; Lozano-Pérez, T.; and Kaelbling,… find
  41. Singh, I.; Blukis, V.; Mousavian, A.; Goyal, A.; Xu, D.; Tremblay, J.; Fox, D.; Thomason, J.; and Garg, A. 2022. Progprompt: Generating situated robot task plans using large language models. arXiv preprint arXiv:2209.11302.
  42. Sohrabi, S.; Riabov, A. V.; Udrea, O.; and Hassanzadeh, O. 2016. Finding diverse high-quality plans for hypothesis generation. In ECAI 2016, 1581–1582. find
  43. IOS Press. Srivastava, S. 2011. Foundations and applications of generalized planning. AI Communications, 24(4): 349–351. find
  44. Srivastava, S.; Immerman, N.; Zilberstein, S.; and Zhang, T. 2011. Directed Search for Generalized Plans Using Classical Planners. In ICAPS. find
  45. Sutton, R. S.; Modayil, J.; Delp, M.; Degris, T.; Pilarski, P. M.; White, A.; and Precup, D. 2011. Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction. In The 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, 76… find
  46. Takayuki, Y. 2000. On the NP-completeness of the Slither Link puzzle. find
  47. IPSJ SIGNotes ALgorithms. Valmeekam, K.; Olmo, A.; Sreedharan, S.; and Kambhampati, S. 2022. Large Language Models Still Can’t Plan (A Benchmark for LLMs on Planning and Reasoning about Change). arXiv preprint arXiv:2206.10498.
  48. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Chi, E.; Le, Q.; and Zhou, D. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903.
  49. Winner, E. Z. 2008. Learning Domain-Specific Planners from Example Plans. Ph.D. thesis, Carnegie Mellon University, USA. find
  50. Xia, C. S.; and Zhang, L. 2023. Conversational Automated Program Repair. arXiv:2301.13246. — [arXiv:2301.13246](https://arxiv.org/abs/2301.13246) find
  51. Xie, Y.; Yu, C.; Zhu, T.; Bai, J.; Gong, Z.; and Soh, H. 2023. Translating natural language to planning goals with large-language models. arXiv preprint arXiv:2302.05128. — [arXiv:2302.05128](https://arxiv.org/abs/2302.05128) find
  52. Yang, R.; Silver, T.; Curtis, A.; Lozano-Perez, T.; and Kaelbling, L. P. 2022. PG3: Policy-Guided Planning for Generalized Policy Generation. In IJCAI. find
  53. Zheng, W.; Sharan, S.; Jaiswal, A. K.; Wang, K.; Xi, Y.; Xu, D.; and Wang, Z. 2023. Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generation. arXiv preprint arXiv:2305.00909. — [arXiv:2305.00909](https://arxiv.org/abs/2305.00909) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/silver-2023-generalized-planning-pddl-domains-pretrained/