Elamur Research
☰
home / papers / 2022 / TALM: Tool Augmented Language Models

TALM: Tool Augmented Language Models

Written for the Research library (claude-opus-5-5) from the paper text. Original — arxiv.org/pdf/2205.12255

Summary

Bigger language models memorize more facts, but scale cannot give them information that is new, private, or missing from the training data, and it cannot let them act through APIs. TALM addresses this by connecting a model to external, non-differentiable tools using only text. The model writes a query for a tool, and a delimiter triggers the call. The tool's reply is appended to the sequence, and the model then writes its final answer.

Labelled tool-use traces are scarce, so the authors grow them through iterative self-play. Starting from a small set of demonstrations, the finetuned model samples many tool calls over ordinary task data. It keeps only the trajectories whose answers match the gold target and retrains on the enlarged set.

The authors tested T5 checkpoints on Natural Questions with a BM25 retriever and on MathQA with a formula solver. Tool-augmented models beat plain LMs at every size, and small TALMs outperformed far larger baselines. This suggests that tools can partly replace raw scale.

Key points

  • Backbones are pretrained T5 models in base (220M), large (770M) and XL (3B) sizes, set up as Seq2Seq. During self-play the model samples with temperature 1.0 and top-k 40, producing up to 600 sequences per example. Evaluation uses beam search with 4 beams.
  • Natural Questions (≈300k training examples) was used in its short-answer setting, with boolean questions removed. The oracle Wikipedia contexts were not given to the model directly. Instead, they populated a BM25 index that serves as the retrieval tool. Starting from 150 demonstrations and two self-play rounds, the 220M TALM outperformed the 3B closed-book LM.
  • MathQA (≈30k examples) was paired with a solver that executes formulas. About 70% of the dataset's formulas reproduce the correct answer. The bootstrap set is a 10% sample of the valid formulas, roughly 2k examples.
  • In ablations, one round of self-play clearly beats training on the bootstrap set alone. Gains continue through three rounds for all model sizes.
  • In out-of-distribution tests, the authors swapped the BM25 retriever for a public search engine. This let TALM answer questions about recent topics such as Wordle. The solver also lets TALM handle large-number arithmetic, where the plain LM fails.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

110 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 93 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2205.12255 · arXiv 2205.12255

@misc{parisi2022talm,
  title = {{TALM: Tool Augmented Language Models}},
  author = {Parisi, Aaron and Zhao, Yao and Fiedel, Noah},
  year = {2022},
  doi = {10.48550/arxiv.2205.12255},
  eprint = {2205.12255},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2205.12255}
}

References

20 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J Joshi, Rya… find our card
  2. Aida Amini, Saadia Gabriel, Peter Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation-based formalisms, 2019. URL https://arxiv.org/abs/1905.13319. ↗ our card
  3. Daniel Andor, Luheng He, Kenton Lee, and Emily Pitler. Giving bert a calculator: Finding operations and arguments with reading comprehension, 2019. find
  4. Thomas Anthony, Zheng Tian, and David Barber. Thinking fast and slow with deep learning and tree search. CoRR, abs/1705.08439, 2017. URL http: //arxiv.org/abs/1705.08439. find
  5. BIG-bench collaboration. Beyond the imitation game: Measuring and extrapolating the capabilities of language models. In preparation, 2021. URL https: //github.com/google/BIG-bench/. find
  6. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cas… find
  7. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  8. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models, 2022. URL https://arxiv. org/abs/2202.07646. find
  9. Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling, 2021. URL https://arxiv.org/abs/ 2106.01345. ↗
  10. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… find
  11. Paul F. Christiano, Buck Shlegeris, and Dario Amodei. Supervising strong learners by amplifying weak experts. CoRR, abs/1810.08575, 2018. URL http: //arxiv.org/abs/1810.08575. find
  12. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Realm: Retrieval-augmented language model pre-training, 2020. find
  13. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. CoRR, abs/2103.03874, 2021. URL https://arxiv. org/abs/2103.03874. find our card
  14. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red-field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural quest… find our card
  15. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks, 2020. URL https:// arxiv.org/abs/2005.1140… find
  16. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2019. URL https://arxiv.org/ abs/1910.10683. ↗
  17. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model?, 2020. URL https: //arxiv.org/abs/2002.08910. find
  18. Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, et al. Scaling up models and data with t5x and seqio. arXiv preprint arXiv:2203.17189, 2022. find
  19. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2017. find
  20. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems, 2019. URL https://arxiv. org/abs/1905.00537. find our card