Elamur Research⌕
☰
home / papers / 2023 / PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning

PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning

Paper · 2023 · Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin et al.

Summary

The paper asks whether compact language models can handle procedural planning, meaning turning a high-level goal into an ordered list of steps. This matters because relying on huge proprietary LLMs is expensive and hard to reproduce. The authors propose PlaSma, which has two parts. First, symbolic procedural knowledge distillation: a GPT-3 teacher is few-shot prompted to produce goals, plans, conditions (such as missing equipment or location limits), and plans rewritten to fit those conditions. The outputs are filtered by RoBERTa-Large critics, yielding the CoPlan dataset. Students ranging from 770M to 11B parameters are then trained on it, either per task or jointly. Second, at inference time, a verifier scores each candidate step inside a step-wise beam search (PlaSma+), which improves ordering and coherence. The paper also introduces a new task, counterfactual replanning: revising an existing plan when a constraint appears. The distilled students match or beat their much larger teachers, which shows that small, open models can serve as practical planners, including in embodied settings.

Key points

  • CoPlan began as about 140k generated goal–plan pairs. After critic filtering it keeps 57,794 (goal, plan) pairs and 43,690 tuples for constrained and counterfactual tasks. It is roughly 11× larger than ProScript/DeScript, with about 74% precision.
  • The critics are RoBERTa-Large binary classifiers trained on 13K Amazon Mechanical Turk judgments. Their rejection thresholds are 0.65 for plans, 0.76 for conditions, and 0.82 for counterfactual plans.
  • On standard planning, students improve on their teacher by 17.57% (relative, on average). The best student is comparable to a GPT-3 model 16× its size.
  • In human evaluation, constrained plans are judged valid 93% of the time and counterfactual replans 86% of the time.
  • In VirtualHome, PlaSma outperforms the GPT-3-based approach of Huang et al. (2022) by 17 points absolute on executability and 25 points on correctness. Data and code are released at github.com/allenai/PlaSma.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WashingtonSeattle · United States402 works · 212,762 citations🔬Allen Institute for Artificial IntelligenceSeattle · United States304 works · 106,138 citations🎓University of Southern CaliforniaLos Angeles · United States175 works · 37,047 citations🎓University of PittsburghPittsburgh · United States65 works · 23,401 citations🎓Tohoku UniversitySendai · Japan23 works · 595 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, Jena D. Hwang, Xiang Lorraine Li, Hirona J. Arai, Soumya Sanyal, Keisuke Sakaguchi, Xiang Ren and Yejin Choi. 2023. PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning. arXiv:2305.19472.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2305.19472 · arXiv 2305.19472

@misc{brahman2023plasma,
  title = {{PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning}},
  author = {Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, Jena D. Hwang, Xiang Lorraine Li, Hirona J. Arai, Soumya Sanyal, Keisuke Sakaguchi, Xiang Ren and Yejin Choi},
  year = {2023},
  doi = {10.48550/arxiv.2305.19472},
  eprint = {2305.19472},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.19472}
}

References

52 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil… — Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
  2. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pp. 610–623, New York, NY, USA, 2021. Associati… — [doi:10.1145/3442188.3445922](https://doi.org/10.1145/3442188.3445922) find
  3. Chandra Bhagavatula, Jena D. Hwang, Doug Downey, Ronan Le Bras, Ximing Lu, Lianhui Qin, Keisuke Sakaguchi, Swabha Swayamdipta, Peter West, and Yejin Choi. I2d2: Inductive knowledge distillation with neurologic and self-imitation, 2023. find
  4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
  5. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf. ↗
  6. Nathanael Chambers and Dan Jurafsky. Unsupervised learning of narrative event chains. In Proceedings of ACL-08: HLT, pp. 789–797, Columbus, Ohio, June 2008. Association for Computational Linguistics. URL https://aclanthology.org/P08-1090. ↗
  7. Katherine M Collins, Catherine Wong, Jiahai Feng, Megan Wei, and Josh Tenenbaum. Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning tasks. In Proceedings of the Annual Meeting of the Cognitive Science S… find
  8. Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ondřej Dušek, Chris Chinenye Emezue, Varun Gangal, Cristina Garbacea, Tatsunori… — The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
  9. Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati. Leveraging pre-trained large language models to construct and utilize world models for model-based task planning, 2023.
  10. Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. ArXiv, abs/1503.02531, 2015.
  11. Cong Duy Vu Hoang, Gholamreza Haffari, and Trevor Cohn. Towards decoding as continuous optimisation in neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 146–156, Copenhagen, Denmark, September 2017. Association for Computationa… find
  12. Chris Hokamp and Qun Liu. Lexically constrained decoding for sequence generation using grid beam search. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1535–1546, Vancouver, Canada, July 2017. Association for Computational Ling…
  13. Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learn…
  14. PMLR, 2022. URL https://proceedings.mlr.press/v162/huang22a.html. ↗
  15. Peter Jansen. Visually-grounded planning without vision: Language models infer detailed plans from high-level instructions. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 4412–4417, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020… find
  16. Subbarao Kambhampati, Sarath Sreedharan, Mudit Verma, Yantian Zha, and Lin Guan. Symbols as a lingua franca for bridging human-ai chasm for explainable and advisable ai systems, 2021. find
  17. Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. GeDi: Generative discriminator guided sequence generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 4929–4952, Punta Cana, Dominican…
  18. Sachin Kumar, Eric Malmi, Aliaksei Severyn, and Yulia Tsvetkov. Controlled text generation as continuous optimization with multiple constraints. In Neural Information Processing Systems, 2021. find
  19. Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013. ↗
  20. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692, 2019a.
  21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019b.
  22. Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. NeuroLogic decoding: (un)supervised neural text generation with predicate logic constraints. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:…
  23. Ximing Lu, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah A. Smith, and Yejin Choi. NeuroLogic a*esque decoding: Constrained text generation with lookahead heuristics. In Proceedings of the 2022 Conference of the Nort…
  24. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 808…
  25. Yujie Lu, Weixi Feng, Wanrong Zhu, Wenda Xu, Xin Eric Wang, Miguel Eckstein, and William Yang Wang. Neuro-symbolic procedural planning with commonsense prompting. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=iOc57X9KM54. ↗
  26. Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, and Graham Neubig. Language models of code are few-shot commonsense learners. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 1384–1403, Abu Dhabi, United Arab Emirates, December 2022. Association for Com…
  27. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4, 2023.
  28. OpenAI. Openai api pricing, 2023. URL https://openai.com/pricing. Accessed: 2023-05-15. ↗
  29. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318, Philadelphia, Pennsylvania, USA, July 2002. Association for Computatio…
  30. Damian Pascual, Béni Egressy, Florian Bolli, and Roger Wattenhofer. Directed beam search: Plug-and-play lexically constrained language generation. ArXiv, abs/2012.15416, 2020. — [arXiv:2012.15416](https://arxiv.org/abs/2012.15416) find
  31. Douglas Pearson and John Laird. Incremental learning of procedural planning knowledge in challenging environments. Computational Intelligence, 21:414–439, 11 2005. doi: 10.1111/j.1467-8640. 2005.00280.x. — [doi:10.1111/j.1467-8640](https://doi.org/10.1111/j.1467-8640) find
  32. Ethan Perez, Douwe Kiela, and Kyunghyun Cho. True few-shot learning with language models. NeurIPS, 2021. URL https://arxiv.org/abs/2105.11447. ↗
  33. Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8494–8502, 2018. find
  34. Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut, and Yejin Choi. Back to the future: Unsupervised backprop-based decoding for counterfactual and abductive commonsense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in…
  35. Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. Cold decoding: Energy-based constrained text generation with langevin dynamics. Advances in Neural Information Processing Systems, 2022. find
  36. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. 2018. find
  37. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. URL http://jmlr.org/papers/v21/20… ↗
  38. Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, and Yejin Choi. proScript: Partially ordered scripts generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 2138–2149, Punta Cana, Dominican Republic, November 2021. Association for C… find
  39. Abhilasha Sancheti and Rachel Rudinger. What do large language models learn about scripts? In Proceedings of the 11th Joint Conference on Lexical and Computational Semantics, pp. 1–11, Seattle, Washington, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/ 2022.starsem-1.1. URL… find
  40. Roger C. Schank and Robert P. Abelson. Scripts, plans and knowledge. In International Joint Conference on Artificial Intelligence, 1975a. find
  41. Roger C. Schank and Robert P. Abelson. Scripts, plans and knowledge. In PN Johnson-Laird and PC Wason (eds.), Thinking: Readings in Cognitive Science, Proceedings of the Fourth International Joint Conference on Artificial Intelligence, pp. 151–157. Tbilisi, USSR, 1975b. find
  42. Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. Large language models still can’t plan (a benchmark for llms on planning and reasoning about change), 2023. find
  43. Lilian D. A. Wanzare, Alessandra Zarcone, Stefan Thater, and Manfred Pinkal. A crowdsourced database of event sequence descriptions for the acquisition of high-quality script knowledge. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pp. 3494–3501… find
  44. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum? id=gEZrGCozdqR. ↗
  45. Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. Symbolic knowledge distillation: from general language models to commonsense models. In Proceedings of the 2022 Conference of the North American Chapter of the Association f…
  46. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… — Transformers: State-of-the-Art Natural Language Processing
  47. Te-Lin Wu, Alex Spangher, Pegah Alipoormolabashi, Marjorie Freedman, Ralph Weischedel, and Nanyun Peng. Understanding multimodal procedural knowledge by sequencing multimodal instructional manuals. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:… find
  48. Kevin Yang and Dan Klein. FUDGE: Controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2021. doi: 10.18653/v1…
  49. Siyu Yuan, Jiangjie Chen, Ziquan Fu, Xuyang Ge, Soham Shah, Charles Jankowski, Yanghua Xiao, and Deqing Yang. Distilling script knowledge from large language models for constrained language planning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1… find
  50. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. find
  51. OpenReview.net, 2020. URL https://openreview.net/forum?id=SkeHuCVFDr. ↗
  52. Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large-scale task planning, 2023. *Figure 5: Goal diversity in C O P LAN* *Figure 6: Condition diversity in C O P LAN*

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/brahman-2023-plasma-making-small-language-models/