Elamur Research
☰
home / papers / 2023 / Tool Learning with Foundation Models

Tool Learning with Foundation Models

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2304.08354

Summary

This survey asks how large pretrained models can learn to operate external tools such as search engines, calculators, databases, image generators and robots. It also argues that earlier supervised and reinforcement-learning systems were too weak to handle tools reliably. The authors first cover background: how tool use arose in human cognition, a classification of tools by the kind of interface they expose, and how tools and foundation models complement each other. They then review existing research and organise it into one framework. A controller model interprets the user's request, splits it into subtasks, revises its plan by reasoning over feedback, and chooses a suitable tool for each step. The survey also covers training from demonstrations and from feedback, and ways to make tool skills transfer to unseen tools. In the empirical part, the authors test current models on 18 tools and find that they can already use tools competently. The paper closes with open questions on safety, AI-created tools, personalisation and knowledge conflicts. It serves as a reference map for building tool-using LLM agents.

Key points

  • The paper names the paradigm "tool learning with foundation models" and argues that models plus tools beat either one alone on accuracy, efficiency and automation.
  • In its general framework, the controller model understands intent, decomposes tasks, plans adaptively through reasoning, and selects tools for each subtask.
  • Training strategies covered: learning from demonstrations, learning from feedback (environment or human), and generalisation to new tools.
  • The evaluation covers 18 tools, with case studies including Wikipedia, maps, weather, translation, database, table processing, knowledge graphs, ALFWorld, stock data, online shopping, AI painting and slide making. The code is released as BMTools (github.com/OpenBMB/BMTools).
  • Open directions: safe and trustworthy tool use, tools for large complex systems, models that create tools, personalised and embodied tool learning, conflicts between a model's internal knowledge and tool outputs, and tool use for scientific discovery.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

199 papers of our library cite it (of the 140673 whose bibliographies we read); 23 of them have a page here, 176 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2304.08354 · arXiv 2304.08354

@misc{qin2023tool,
  title = {{Tool Learning with Foundation Models}},
  author = {Qin, Yujia and Hu, Shengding and Lin, Yankai and Chen, Weize and Ding, Ning and Cui, Ganqu and Zeng, Zheni and Huang, Yufei and Xiao, Chaojun and Han, Chi and Fung, Yi Ren and Su, Yusheng and Wang, Huadong and Qian, Cheng and Tian, Runchu and Zhu, Kunlun and Liang, Shihao and Shen, Xingyu and Xu, Bokai and Zhang, Zhen and Ye, Yining and Li, Bowen and Tang, Ziwei and Yi, Jing and Zhu, Yuzhang and Dai, Zhenning and Yan, Lan and Cong, Xin and Lu, Yaxi and Zhao, Weilin and Huang, Yuxiang and Yan, Junxi and Han, Xu and Sun, Xian and Li, Dahai and Phang, Jason and Yang, Cheng and Wu, Tongshuang and Ji, Heng and Liu, Zhiyuan and Sun, Maosong},
  year = {2023},
  doi = {10.48550/arxiv.2304.08354},
  eprint = {2304.08354},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.08354}
}

References

222 references, 53 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Abramson, Arun Ahuja, Federico Carnevale, Petko Georgiev, Alex Goldin, Alden Hung, Jessica Landon, Jirka Lhotka, Timothy Lillicrap, Alistair Muldal, et al. Improving multimodal interactive agents with reinforcement learning from human feedback. arXiv preprint arXiv:2211.11602, 2022. — [arXiv:2211.11602](https://arxiv.org/abs/2211.11602) find
  2. Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. Do as i can, not as i say: Grounding language in robotic affordances. ArXiv preprint, abs/2204.01691, 2022. URL https://arxiv.org/abs/2204.01… ↗ our card
  3. Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik’s cube with a robot hand. ArXiv preprint, abs/1910.07113, 2019. URL https://arxiv.org/abs/1910.07113. ↗
  4. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. ArXiv preprint, abs/2204.14198, 2022. URL https://arxiv.org/abs/2204.14198. ↗ our card
  5. Kelsey R Allen, Kevin A Smith, and Joshua B Tenenbaum. Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning. ArXiv preprint, abs/1907.09620, 2019. URL https: //arxiv.org/abs/1907.09620. — [arXiv:1907.09620](https://arxiv.org/abs/1907.09620) find
  6. Stanley H Ambrose. Coevolution of composite-tool technology, constructive memory, and language: implications for the evolution of modern human behavior. Current Anthropology, 51(S1):S135–S147, 2010. find
  7. Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Lin… find our card
  8. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul F. Christiano, John Schulman, and Dan Mané. Concrete problems in AI safety. ArXiv preprint, abs/1606.06565, 2016. URL https://arxiv.org/abs/1606.06565. ↗ our card
  9. Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V. Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, Zaid Alyafeai, Manan Dey, Andrea Santilli, Zhiqing Sun, Srulik Ben-david, Canwen Xu, Gunjan Chhablani, Han Wang, Jason Fries, Maged Al-shaibani, Shanya… find our card
  10. Michael Bain and Claude Sammut. A framework for behavioural cloning. In Machine Intelligence 15, pp. 103–129, 1995. URL http://www.cse.unsw.edu.au/~claude/papers/MI15.pdf. ↗
  11. Bowen Baker, Ingmar Kanitscheider, Todor M. Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. Emergent tool use from multi-agent autocurricula. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. find
  12. OpenReview.net, 2020. URL https://openreview.net/forum?id=SkxpxJBKwS. ↗
  13. Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. ArXiv preprint, abs/2206.11795, 2022. URL https://arxiv.org/abs/2206.11795. — [arXiv:2206.11795](https://arxiv.org/abs/2206.11795) ↗
  14. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Andrea Pohoreckyj Danyluk, Léon Bottou, and Michael L. Littman (eds.), Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, June 14-18, 2009, volum… find our card
  15. ACM, 2009. doi: 10.1145/1553374.1553380. URL https://doi.org/10.1145/1553374.1553380. — [doi:10.1145/1553374.1553380](https://doi.org/10.1145/1553374.1553380) ↗
  16. Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław D˛ebiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforcement learning. ArXiv preprint, abs/1912.06680, 2019. URL https://arxiv.org/abs/1912.06680. ↗
  17. Christophe Boesch, Daša Bombjaková, Amelia Meier, and Roger Mundry. Learning curves and teaching when acquiring nut-cracking in humans and chimpanzees. Scientific Reports, 9(1):1515, 2019. find
  18. Daniil A. Boiko, Robert MacKnight, and Gabe Gomes. Emergent autonomous scientific research capabilities of large language models, 2023. find
  19. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. ArXiv preprint, abs/2108.07258, 2021. URL https://arxiv.org/abs/2108.07258. ↗ our card
  20. Ali Borji. A categorical archive of chatgpt failures. ArXiv preprint, abs/2302.03494, 2023. URL https: //arxiv.org/abs/2302.03494. find
  21. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  22. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. ArXiv preprint, abs/2303.12712, 2023. URL https://arxiv.org/abs/2303.1271… ↗ our card
  23. Benjamin Burger, Phillip M Maffettone, Vladimir V Gusev, Catherine M Aitchison, Yang Bai, Xiaoyan Wang, Xiaobo Li, Ben M Alston, Buyi Li, Rob Clowes, et al. A mobile robotic chemist. Nature, 583(7815): 237–241, 2020. find
  24. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633–2650, 2021. find our card
  25. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. ArXiv preprint, abs/2202.07646, 2022. URL https://arxiv.org/abs/2202.07646. ↗
  26. Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. ArXiv preprint, abs/2301.13188, 2023. URL https://arxiv.org/abs/2301.13188. — [arXiv:2301.13188](https://arxiv.org/abs/2301.13188) ↗
  27. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. ArXiv preprint, abs/2107.03374, 2021. URL https://arxiv.org/abs/2107.03374. ↗ our card
  28. Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. ArXiv preprint, abs/2211.12588, 2022a. URL https://arxiv.org/abs/2211.12588. ↗ our card
  29. Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, and Heng Ji. A close look into the calibration of pre-trained language models. ArXiv preprint, abs/2211.00151, 2022b. URL https://arxiv.org/abs/2211.00151. ↗
  30. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 3… find
  31. Geraldine Clarebout, Jan Elen, Norma A Juarez Collazo, Griet Lust, and Lai Jiang. Metacognition and the use of tools. International handbook of metacognition and learning technologies, pp. 187–195, 2013. find
  32. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. ArXiv preprint, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168. ↗ our card
  33. Felipe Codevilla, Eder Santana, Antonio M. López, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pp. 9328–9337. find
  34. IEEE, 2019. doi: 10.1109/ICCV.2019.00942. URL https://doi.org/10.1109/ICCV.2019.00942. — [doi:10.1109/iccv.2019.00942](https://doi.org/10.1109/iccv.2019.00942) ↗
  35. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. No language left behind: Scaling human-centered machine translation. ArXiv preprint, abs/2207.04672, 2022. URL https://arxiv.org/abs/2207.04… ↗
  36. Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. ArXiv preprint, abs/2205.09712, 2022. URL https://arxiv.org/abs/2205.09712. ↗
  37. Ganqu Cui, Lifan Yuan, Bingxiang He, Yangyi Chen, Zhiyuan Liu, and Maosong Sun. A unified evaluation of textual backdoor learning: Frameworks and benchmarks. Advances in Neural Information Processing Systems, 2022. find
  38. Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de Las Casas, et al. Magnetic control of tokamak plasmas through deep reinforcement learning. Nature, 602(7897):414–419, 2022. find
  39. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volum… find
  40. Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: An embodied multimodal language model. ArXiv preprint, abs/2303.03378, 2023. URL https://arxiv.org/abs/2303.03378. ↗
  41. Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2): 230–244, 2022. find
  42. Andreas K Engel, Alexander Maye, Martin Kurthen, and Peter König. Where’s the action? the pragmatic turn in cognitive science. Trends in cognitive sciences, 17(5):202–209, 2013. find
  43. Jacqueline Fagard, Lauriane Rat-Fischer, Rana Esseily, Eszter Somogyi, and JK O’Regan. What does it take for an infant to learn how to use a tool by observation? Frontiers in psychology, 7:267, 2016. URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4771934/. ↗
  44. Scott H Frey. What puts the how in where? tool use and the divided visual streams hypothesis. Cortex, 43(3): 368–375, 2007. find
  45. Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. Incoder: A generative model for code infilling and synthesis. ArXiv preprint, abs/2204.05999, 2022. URL https://arxiv.org/abs/2204.05999. ↗
  46. Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, et al. Threedworld: A platform for interactive multi-modal physical simulation. arXiv preprint arXiv:2007.04954, 2020. — [arXiv:2007.04954](https://arxiv.org/abs/2007.04954) find
  47. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. ArXiv preprint, abs/2211.10435, 2022. URL https: //arxiv.org/abs/2211.10435. find
  48. Kathleen R Gibson, Kathleen Rita Gibson, and Tim Ingold. Tools, language and cognition in human evolution. Cambridge University Press, 1993. find
  49. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Retrieval augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 39… find
  50. PMLR, 2020. URL http://proceedings.mlr.press/v119/guu20a.html. ↗
  51. Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Liang Zhang, Wentao Han, Minlie Huang, Qin Jin, Yanyan Lan, Yang Liu, Zhiyuan Liu, Zhiwu Lu, Xipeng Qiu, Ruihua Song, Jie Tang, Ji-Rong Wen, Jinhui Yuan, Wayne Xin Zhao, and Jun Zhu. Pre-trained models: Past, present and… — [doi:10.1016/j.aiopen.2021.08.002](https://doi.org/10.1016/j.aiopen.2021.08.002) find
  52. Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenyà, Pieter Abbeel, Alexei A. Efros, Lerrel Pinto, and Xiaolong Wang. Self-supervised policy adaptation during deployment. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. find
  53. OpenReview.net, 2021. URL https://openreview.net/forum?id=o\_V-MjyyGV\_. ↗
  54. Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, and Furu Wei. Language models are general-purpose interfaces. ArXiv preprint, abs/2206.06336, 2022. URL https://arxiv.org/abs/2206.06336. — [arXiv:2206.06336](https://arxiv.org/abs/2206.06336) ↗
  55. Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety. ArXiv preprint, abs/2109.13916, 2021. URL https://arxiv.org/abs/2109.13916. ↗
  56. Mikolaj Hernik and Gergely Csibra. Functional understanding facilitates learning about tools in human children. Current Opinion in Neurobiology, 19(1):34–38, 2009. ISSN 0959-4388. doi: https://doi.org/10.1016/j.conb.2009.05.003. URL https://www.sciencedirect.com/science/article/pii/S0959438809000415… — [doi:10.1016/j.conb.2009.05.003](https://doi.org/10.1016/j.conb.2009.05.003) ↗
  57. Cecilia Heyes. Cognitive gadgets: The cultural evolution of thinking. Harvard University Press, 2018. find
  58. Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato (eds.), International Conference on Machine Learn… find our card
  59. PMLR, 2022a. URL https://proceedings.mlr. press/v162/huang22a.html. ↗
  60. Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. ArXiv preprint, abs/2207.05608, 2022b. URL https://arxiv.org/abs/2207.05608. ↗
  61. Gavin R Hunt. Manufacture and use of hook-tools by new caledonian crows. Nature, 379(6562):249–251, 1996. find
  62. Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne. Imitation learning: A survey of learning methods. ACM Computing Surveys (CSUR), 50(2):1–35, 2017. URL https://dl.acm.org/doi/abs/10.1145/3054912?casa\_token=DlqMmdYdq8sAAAAA: 2LEBqNJ9is6JtifUJCcvoUUiKu8zj75-Su3PBEDpmHqX1-PtZFKHv1SPf… — [doi:10.1145/3054912?casa\_token=dlqmmdydq8saaaaa](https://doi.org/10.1145/3054912?casa_token=dlqmmdydq8saaaaa) ↗
  63. Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Dániel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, et al. Opt-iml: Scaling language model instruction meta learning through the lens of generalization. ArXiv preprint, abs/2212.12017, 2022. URL https://… find our card
  64. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Few-shot learning with retrieval augmented language models. ArXiv preprint, abs/2208.03299, 2022. URL https://arxiv.org/abs/2208.03299. ↗
  65. Bernard J. Jansen, Danielle L. Booth, and Amanda Spink. Determining the user intent of web search engine queries. In Carey L. Williamson, Mary Ellen Zurko, Peter F. Patel-Schneider, and Prashant J. Shenoy (eds.), Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff, Al… find
  66. ACM, 2007. doi: 10.1145/1242572.1242739. URL https://doi.org/10.1145/1242572.1242739. — [doi:10.1145/1242572.1242739](https://doi.org/10.1145/1242572.1242739) ↗
  67. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial… find our card
  68. AAAI Press, 2020. URL https: //aaai.org/ojs/index.php/AAAI/article/view/6311. find
  69. John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583–589, 2021. URL https://www.nature.com/articles/… ↗ our card
  70. Abhishek Kadian, Joanne Truong, Aaron Gokaslan, Alexander Clegg, Erik Wijmans, Stefan Lee, Manolis Savva, Sonia Chernova, and Dhruv Batra. Sim2real predictivity: Does evaluation in simulation predict real-world performance? IEEE Robotics and Automation Letters, 5(4):6670–6677, 2020. find
  71. Daniel Kahneman. Thinking, fast and slow. macmillan, 2011. find
  72. Immanuel Kant and Jerome B Schneewind. Groundwork for the Metaphysics of Morals. Yale University Press, 2002. find
  73. Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. RealTime QA: What’s the answer right now?, 2022. URL https://arxiv.org/abs/2207.13332. ↗ our card
  74. David Kelley. The art of reasoning: An introduction to logic and critical thinking. WW Norton & Company, 2013. find
  75. Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. find
  76. OpenReview.net, 2020. URL https://openreview.net/forum?id=HklBjCEKvH. ↗
  77. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Decomposed prompting: A modular approach for solving complex tasks. ArXiv preprint, abs/2210.02406, 2022. URL https://arxiv.org/abs/2210.02406. ↗
  78. Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A Hale. Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback. ArXiv preprint, abs/2303.05453, 2023. URL https://arxiv.org/abs/2303.05453. ↗
  79. Jürgen Kletti. Manufacturing Execution Systems—MES. Springer, 2007. find
  80. Kwang Hyun Ko. Origins of human intelligence: The chain of tool-making and brain evolution. Anthropological Notebooks, 22(1), 2016. find
  81. Sandra Kübler, Ryan McDonald, and Joakim Nivre. Dependency parsing. Synthesis lectures on human language technologies, 1(1):1–127, 2009. find
  82. Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pretrained models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2793–2806, Online, 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.249. URL htt… find
  83. Susanne P Lajoie and Sharon J Derry. Computer environments as cognitive tools for enhancing learning. In Computers as cognitive tools, pp. 269–296. Routledge, 2013. find
  84. Guillaume Lample, Timothee Lacroix, Marie-Anne Lachaux, Aurelien Rodriguez, Amaury Hayat, Thibaut Lavril, Gabriel Ebner, and Xavier Martinet. Hypertree proof search for neural theorem proving. Advances in Neural Information Processing Systems, 35:26337–26349, 2022. find our card
  85. Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. Internet-augmented language models through few-shot prompting for open-domain question answering. ArXiv preprint, abs/2203.05115, 2022. URL https://arxiv.org/abs/2203.05115. ↗ our card
  86. Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International journal of robotics research, 37(4-5):421–436, 2018. find
  87. Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. MLQA: Evaluating cross-lingual extractive question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7315–7330, Online, 2020a. Association for Computational Linguis… find our card
  88. Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Hugo Larochelle, Marc’Aurelio Ranza… find
  89. Guoliang Li, Xuanhe Zhou, Shifu Li, and Bo Gao. Qtune: A query-aware database tuning system with deep reinforcement learning. Proceedings of the VLDB Endowment, 12(12):2118–2130, 2019. find
  90. Shuang Li, Xavier Puig, Yilun Du, Clinton Wang, Ekin Akyurek, Antonio Torralba, Jacob Andreas, and Igor Mordatch. Pre-trained language models for interactive decision-making. ArXiv preprint, abs/2202.01771, 2022. URL https://arxiv.org/abs/2202.01771. ↗
  91. Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. ArXiv preprint, abs/2209.07753, 2022a. URL https://arxiv.org/abs/2209.07753. ↗
  92. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. ArXiv preprint, abs/2211.09110, 2022b. URL https://arxiv.org/abs/2211.09110. ↗
  93. Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao, et al. Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis. ArXiv preprint, abs/2303.16434, 2023. URL https://arxiv.org/abs/2303.16434. — [arXiv:2303.16434](https://arxiv.org/abs/2303.16434) ↗
  94. Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214–3252, Dublin, Ireland, 2022. Association for Computational Linguistics.… find our card
  95. Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1):18, 2020. URL https://www.mdpi.com/1099-4300/23/1/18. ↗
  96. Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings… find
  97. OpenReview.net, 2018a. URL https://openreview.net/forum?id=ryTp3f-0-. ↗
  98. Jason Xinyu Liu, Ziyi Yang, Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, and Ankit Shah. Lang2ltl: Translating natural language commands to temporal robot task specification. ArXiv preprint, abs/2302.11649, 2023. URL https://arxiv.org/abs/2302.11649. — [arXiv:2302.11649](https://arxiv.org/abs/2302.11649) ↗
  99. Ruibo Liu, Jason Wei, Shixiang Shane Gu, Te-Yen Wu, Soroush Vosoughi, Claire Cui, Denny Zhou, and Andrew M Dai. Mind’s eye: Grounded language model reasoning through simulation. ArXiv preprint, abs/2210.05359, 2022. URL https://arxiv.org/abs/2210.05359. ↗
  100. YuXuan Liu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Imitation from observation: Learning to imitate behaviors from raw video via context translation. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1118–1125. find
  101. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7052–7063, Online and Punta Cana, Dominican Republic,… find our card
  102. Abdoulaye O Ly and Moulay Akhloufi. Learning to drive by imitation: An overview of deep behavior cloning methods. IEEE Transactions on Intelligent Vehicles, 6(2):195–209, 2020. find
  103. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651, 2023. URL https://arxiv.org/pdf/2303.17651.pdf. ↗
  104. Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. Personalizing dialogue agents via meta-learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5454–5459, Florence, Italy, 2019. Association for Computational Linguistics. doi: 10.18653… find
  105. Alexandra Mateescu and Madeleine Elish. Ai in context: the labor of integrating new technologies. Technical report, Data & Society Research Institute, 2019. find
  106. Michael Matheny, S Thadaney Israni, Mahnoor Ahmed, and Danielle Whicher. Artificial intelligence in health care: The hope, the hype, the promise, the peril. Washington, DC: National Academy of Medicine, 2019. find
  107. Starting Matlab. Matlab. The MathWorks, Natick, MA, 2012. find
  108. Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, and Antoine Bordes. Training millions of personalized dialogue agents. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2775–2779, Brussels, Belgium, 2018. Association for Computational Linguistics. d… find
  109. Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al. Teaching language models to support answers with verified quotes. ArXiv preprint, abs/2203.11147, 2022. URL https://arxiv.org/abs… ↗ our card
  110. Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, et al. Augmented language models: a survey. ArXiv preprint, abs/2302.07842, 2023. URL https://arxiv.org/abs/2302.07842. ↗ our card
  111. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. A diverse corpus for evaluating and developing English math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 975–984, Online, 2020. Association for Computational Linguistics. doi: 10.… find
  112. Paul Michel and Graham Neubig. Extreme adaptation for personalized neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 312–318, Melbourne, Australia, 2018. Association for Computational Linguistics. doi:… find
  113. Dimitris Milakis, Bart Van Arem, and Bert Van Wee. Policy and society related implications of automated driving: A review of literature and directions for future research. Journal of Intelligent Transportation Systems, 21(4):324–348, 2017. find
  114. Shachar Mirkin and Jean-Luc Meunier. Personalized machine translation: Predicting translational preferences. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2019– 2025, Lisbon, Portugal, 2015. Association for Computational Linguistics. doi: 10.18653/v1/… find
  115. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3470–3487, Dublin, Ireland, 2022.… find
  116. Vikas Misra, MI Khan, and UK Singh. Supply chain management systems: architecture, design and vision. Journal of Strategic Innovation and Sustainability, 6(4):96–101, 2010. find
  117. Steven Mithen. The prehistory of the mind: The cognitive origins of art and science. Thames & Hudson Ltd., 1996. find
  118. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. URL https://arxiv.org/pdf/1312.5602.pdf. ↗
  119. David Nadeau and Satoshi Sekine. A survey of named entity recognition and classification. Lingvisticae Investigationes, 30(1):3–26, 2007. find
  120. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. ArXiv preprint, abs/2112.09332, 2021. URL https://arxiv.org/abs/2112.09332. ↗ our card
  121. Ani Nenkova and Kathleen McKeown. A survey of text summarization techniques. Mining text data, pp. 43–76, 2012. find
  122. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. ArXiv preprint, abs/2112.00114, 2021. URL https: //arxiv.o… find our card
  123. OpenAI. OpenAI: Introducing ChatGPT, 2022. URL https://openai.com/blog/chatgpt. ↗
  124. OpenAI. Gpt-4 technical report, 2023. find our card
  125. Guy A Orban and Fausto Caruana. The neural basis of human tool use. Frontiers in psychology, 5:310, 2014. find
  126. François Osiurak and Arnaud Badets. Tool use and affordance: Manipulation-based versus reasoning-based approaches. Psychological review, 123(5):534, 2016. find
  127. François Osiurak and Dietmar Heinke. Looking for intoolligence: A unified framework for the cognitive study of human tool use and technology. American Psychologist, 73(2):169, 2018. find
  128. François Osiurak and Emanuelle Reynaud. The elephant in the room: what matters cognitively in cumulative technological culture. Behavioral and Brain Sciences, 43:e156, 2020. find
  129. François Osiurak, Mathieu Lesourd, Ludovic Delporte, and Yves Rossetti. Tool use and generalized motor programs: we all are natural born poly-dexters. Scientific Reports, 8(1):10429, 2018. find
  130. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. ArXiv preprint, abs/2203.02155, 2022. URL https://arxiv.org/abs/2203.02155. ↗
  131. Aaron Parisi, Yao Zhao, and Noah Fiedel. Talm: Tool augmented language models. ArXiv preprint, abs/2205.12255, 2022. URL https://arxiv.org/abs/2205.12255. ↗ our card
  132. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023. URL https://arxiv.org/pdf/2304.03442.pdf. ↗
  133. Fábio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. ArXiv preprint, abs/2211.09527, 2022. URL https://arxiv.org/abs/2211.09527. ↗ our card
  134. Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Dmytro Okhonko, Samuel Broscheit, Gautier Izacard, Patrick Lewis, Barlas Oğuz, Edouard Grave, Wen-tau Yih, et al. The web is your oyster–knowledge-intensive nlp against a very large web corpus. ArXiv preprint, abs/2112.09924, 2021. URL https: //… — [arXiv:2112.09924](https://arxiv.org/abs/2112.09924) find
  135. Dean A Pomerleau. Alvinn: An autonomous land vehicle in a neural network. Advances in neural information processing systems, 1, 1988. find
  136. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. ArXiv preprint, abs/2210.03350, 2022. URL https://arxiv.org/abs/2210.03350. ↗
  137. Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018. find
  138. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1–140:67, 2020. URL http://jmlr.org/papers/v21/20-074.html. ↗
  139. Santhosh K Ramakrishnan, Dinesh Jayaraman, and Kristen Grauman. An exploration of embodied visual exploration. International Journal of Computer Vision, 129:1616–1649, 2021. find
  140. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. ArXiv preprint, abs/2204.06125, 2022. URL https://arxiv.org/abs/2204.06125. ↗ our card
  141. Siddharth Reddy, Anca D. Dragan, and Sergey Levine. SQIL: imitation learning via reinforcement learning with sparse rewards. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. find
  142. OpenReview.net, 2020. URL https://openreview.net/forum?id=S1xKd24twB. ↗
  143. Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al. A generalist agent. ArXiv preprint, abs/2205.06175, 2022. URL https://arxiv.org/abs/2205.06175. ↗
  144. Emanuelle Reynaud, Jordan Navarro, Mathieu Lesourd, and François Osiurak. To watch is to work: a review of neuroimaging data on tool use observation network. Neuropsychology Review, 29:484–497, 2019. find
  145. Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, and Jason Weston. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Lingui… find our card
  146. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684–10695, 2022. find
  147. Kevin Roose. A conversation with bing’s chatbot left me deeply https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html, 2023. ↗
  148. Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonatha… find our card
  149. OpenReview.net, 2022. URL https://openreview.net/forum?id=9Vrb9D0WI4. ↗
  150. Fumihiro Sasaki and Ryota Yamashina. Behavioral cloning from noisy demonstrations. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. find
  151. OpenReview.net, 2021. URL https://openreview.net/forum?id=zrT3HcsWSAt. ↗
  152. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. ArXiv preprint, abs/2302.04761, 2023. URL https://arxiv.org/abs/2302.04761. ↗ our card
  153. Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588(7839):604–609, 2020. find
  154. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv preprint, abs/1707.06347, 2017. URL https://arxiv.org/abs/1707.06347. ↗
  155. Amanda Seed and Richard Byrne. Animal tool-use. Current biology, 20(23):R1032–R1039, 2010. find
  156. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface, 2023. find
  157. Tianlin Shi, Andrej Karpathy, Linxi Fan, Jonathan Hernandez, and Percy Liang. World of bits: An open-domain platform for web-based agents. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 201… find
  158. PMLR, 2017. URL http://proceedings.mlr.press/v70/shi17a.html. ↗
  159. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew J. Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. find our card
  160. OpenReview.net, 2021. URL https://openreview.net/forum?id=0IOX0YcCdTn. ↗
  161. Robert W Shumaker, Kristina R Walkup, and Benjamin B Beck. Animal tool behavior: the use and manufacture of tools by animals. JHU Press, 2011. find
  162. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 3784–3803, Punta Cana, Dominican Republic, 2021. Association for Computational Linguistics… find our card
  163. Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, et al. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage. ArXiv preprint, abs/2208.03188, 2022. URL https://arxiv.org/abs/2208… ↗ our card
  164. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140–1144, 2018. find
  165. Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task plans using large language models. ArXiv preprint, abs/2209.11302, 2022. URL https://arxiv.org/abs/2209.11302. ↗ our card
  166. Linda Smith and Michael Gasser. The development of embodied cognition: Six lessons from babies. Artificial life, 11(1-2):13–29, 2005. find
  167. Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Llm-planner: Few-shot grounded planning for embodied agents with large language models. ArXiv preprint, abs/2212.04088, 2022. URL https://arxiv.org/abs/2212.04088. ↗ our card
  168. Haoyu Song, Yan Wang, Kaiyan Zhang, Wei-Nan Zhang, and Ting Liu. BoB: BERT over BERT for training persona-based dialogue models from limited personalized data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Na… find
  169. Robert J Sternberg. The theory of successful intelligence. Review of General psychology, 3(4):292–316, 1999. find
  170. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. Learning to summarize with human feedback. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neu… find
  171. Gita Sukthankar, Christopher Geib, Hung Bui, David Pynadath, and Robert P Goldman. Plan, activity, and intent recognition: Theory and practice. Newnes, 2014. find
  172. Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. Recitation-augmented language models. ArXiv preprint, abs/2210.01296, 2022. URL https://arxiv.org/abs/2210.01296. — [arXiv:2210.01296](https://arxiv.org/abs/2210.01296) ↗
  173. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 20… find our card
  174. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural In… find
  175. Irmgard Teschke, Claudia AF Wascher, Madeleine F Scriba, Auguste MP von Bayern, V Huml, B Siemers, and Sabine Tebbich. Did tool-use evolve with enhanced physical cognitive abilities? Philosophical Transactions of the Royal Society B: Biological Sciences, 368(1630):20120418, 2013. URL https: //www.nc… find
  176. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. ArXiv preprint, abs/2201.08239, 2022. URL https://arxiv.org/abs/2201.08239. ↗ our card
  177. Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 5026–5033. find
  178. IEEE, 2012. URL https://ieeexplore.ieee.org/document/6386109. ↗
  179. Faraz Torabi, Garrett Warnell, and Peter Stone. Behavioral cloning from observation. In Jérôme Lang (ed.), Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pp. 4950–4957. ijcai.org, 2018. doi: 10.24963/ijcai… find
  180. Alexander Matt Turner, Aseem Saxena, and Prasad Tadepalli. Formalizing the problem of side effect regularization. In NeurIPS ML Safety Workshop, 2022. find
  181. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neur… find
  182. Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor. Chatgpt for robotics: Design principles and model abilities. Technical Report MSR-TR-2023-8, Microsoft, February 2023. URL https://www.microsoft.com/en-us/research/publication/chatgpt-for-robotics-design-principles-and-model-abilities/… ↗
  183. Barbara Von Eckardt. What is cognitive science? MIT press, 1995. find
  184. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing… find
  185. Eric Wallace, Tony Zhao, Shi Feng, and Sameer Singh. Concealed data poisoning attacks on NLP models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 139–150, Online, 2021. Association for Computati… find
  186. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox,… find our card
  187. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. find our card
  188. OpenReview.net, 2019b. URL https://openreview.net/forum?id=rJ4km2R5t7. ↗
  189. Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/mesh-transformer-jax, 2021. ↗
  190. Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang, Wei Ye, Xiubo Geng, et al. On the robustness of chatgpt: An adversarial and out-of-distribution perspective. ArXiv preprint, abs/2302.12095, 2023a. URL https://arxiv.org/abs/2302.12095. ↗
  191. Shiguo Wang. A comprehensive survey of data mining-based accounting-fraud detection research. In 2010 International Conference on Intelligent Computation Technology and Automation, volume 1, pp. 50–53. find
  192. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. ArXiv preprint, abs/2203.11171, 2022a. URL https: //arxiv.org/abs/2203.11171. find our card
  193. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. ArXiv preprint, abs/2212.10560, 2022b. URL https://arxiv.org/abs/2212.10560. ↗
  194. Zihao Wang, Shaofei Cai, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560, 2023b. URL https://arxiv.org/pdf/2302.01560.pdf. ↗ our card
  195. Sherwood L Washburn. Tools and human evolution. Scientific American, 203(3):62–75, 1960. find
  196. Peter C Wason. Reasoning about a rule. Quarterly journal of experimental psychology, 20(3):273–281, 1968. find
  197. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. find
  198. OpenReview.net, 2022a. URL https://openreview.net/forum?id=gEZrGCozdqR. ↗
  199. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. ArXiv preprint, abs/2206.07682, 2022b. URL https://arxiv.org/abs/2206.07682. ↗ our card
  200. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. ArXiv preprint, abs/2201.11903, 2022c. URL https://arxiv.org/abs/2201.11903. ↗ our card
  201. David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31–36, 1988. find
  202. Jess Whittlestone, Rune Nyrup, Anna Alexandrova, Kanta Dihal, and Stephen Cave. Ethical and societal implications of algorithms, data, and artificial intelligence: a roadmap for research. London: Nuffield Foundation, 2019. find
  203. Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. ArXiv preprint, abs/2303.04671, 2023. URL https://arxiv.org/abs/2303.04671. ↗ our card
  204. Yuwei Wu, Xuezhe Ma, and Diyi Yang. Personalized response generation via generative split memory network. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1956–1970, Online, 2021. Association for Co… find
  205. Joern Wuebker, Patrick Simianer, and John DeNero. Compact personalized models for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 881–886, Brussels, Belgium, 2018. Association for Computational Linguistics. doi: 10.18653/v1/D… find
  206. Rui Yan, Jian-Yun Nie, and Xiaoming Li. Summarize what you are interested in: An optimization framework for interactive personalized summarization. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 1342–1351, Edinburgh, Scotland, UK., 2011. Association fo… find
  207. Diyi Yang and Lucie Flek. Towards user-centric text-to-text generation: A survey. In Text, Speech, and Dialogue: 24th International Conference, TSD 2021, Olomouc, Czech Republic, September 6–9, 2021, Proceedings 24, pp. 3–22. find
  208. Guang-Zhong Yang, Jim Bellingham, Pierre E Dupont, Peer Fischer, Luciano Floridi, Robert Full, Neil Jacobstein, Vijay Kumar, Marcia McNutt, Robert Merrifield, et al. The grand challenges of science robotics. Science robotics, 3(14):eaar7650, 2018a. find
  209. Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. ArXiv preprint, abs/2303.11381, 2023. URL https://arxiv.org/abs/2303.11381. ↗
  210. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369– 2380,… find our card
  211. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. ArXiv preprint, abs/2207.01206, 2022a. URL https: //arxiv.org/abs/2207.01206. find our card
  212. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. ArXiv preprint, abs/2210.03629, 2022b. URL https://arxiv.org/abs/2210.03629. ↗ our card
  213. Joel Ye, Dhruv Batra, Erik Wijmans, and Abhishek Das. Auxiliary tasks speed up learning point goal navigation. In Conference on Robot Learning, pp. 498–516. PMLR, 2021. find
  214. Licheng Yu, Xinlei Chen, Georgia Gkioxari, Mohit Bansal, Tamara L. Berg, and Dhruv Batra. Multi-target embodied question answering. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 6309–6318. Computer Vision Foundation / IEEE, 2019.… — [doi:10.1109/cvpr.2019.00647](https://doi.org/10.1109/cvpr.2019.00647) find
  215. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Process… find our card
  216. Andy Zeng, Adrian Wong, Stefan Welker, Krzysztof Choromanski, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, et al. Socratic models: Composing zero-shot multimodal reasoning with language. ArXiv preprint, abs/2204.00598, 2022. URL https://arxiv. org/ab… find our card
  217. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2204–2213, Melbourne, Aus… find our card
  218. Hanxun Zhong, Zhicheng Dou, Yutao Zhu, Hongjin Qian, and Ji-Rong Wen. Less is more: Learning to refine dialogue history for personalized dialogue generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… find
  219. Shuyan Zhou, Uri Alon, Frank F. Xu, Zhiruo Wang, Zhengbao Jiang, and Graham Neubig. Docprompting: Generating code by retrieving the docs, 2023. find
  220. Xuanhe Zhou, Chengliang Chai, Guoliang Li, and Ji Sun. Database meets artificial intelligence: A survey. IEEE Transactions on Knowledge and Data Engineering, 34(3):1096–1116, 2020. find
  221. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. ArXiv preprint, abs/1909.08593, 2019. URL https://arxiv.org/abs/1909.08593. ↗
  222. A. Case Study A A Case Case Study Study find