Elamur Research
☰
home / papers / 2021 / Show your work: Scratchpads for intermediate computation with language models

Show your work: Scratchpads for intermediate computation with language models

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2112.00114

Summary

Large pre-trained Transformers handle single-pass tasks well, such as writing text or code. They break down on problems needing many sequential steps, like adding long numbers or predicting what a program will output. The authors trace this partly to the fixed amount of computation available in one forward pass. Rather than changing the architecture, they change the task format. The model first writes out its intermediate steps as tokens in a "scratchpad" and only then gives the final answer. Training uses ordinary supervised likelihood on these textual traces.

They test this with dense decoder-only language models (2M to 137B parameters, pre-trained on web and dialog data) on three tasks of rising generality: long addition, polynomial evaluation, and Python execution prediction via line-by-line traces of local variable states. In every setting, scratchpads clearly beat direct answer prediction, including generalization to longer inputs than seen in training. The work shows that intermediate textual computation can unlock algorithmic reasoning in existing models. It foreshadows later "step-by-step" prompting methods.

Key points

  • Addition setup: models were fine-tuned on 100k problems with 1–8 digit operands (5k steps, batch size 32). They were tested on 10k in-distribution examples and 1k examples each for 9- and 10-digit inputs, with model sizes from 2M to 1B parameters.
  • Addition results:
    • Above a critical model size, scratchpad models solve the task, while no-scratchpad baselines fail even at the largest size.
    • On out-of-distribution lengths, the baselines collapse completely, whereas scratchpad accuracy keeps rising with scale.
  • Other tasks: scratchpads also help polynomial evaluation, in both few-shot and fine-tuning regimes. For program execution, emitting full traces annotated with variable states greatly improves prediction of a program's output.
  • Ablation: training on scratchpads first and then fine-tuning for direct answers gives no notable gain. This suggests the extra supervision signal alone does not explain the improvement.
  • Effect on synthesis: the trace-trained "MBPP-aug + CodeNet + single line" model reaches 54% on few-shot MBPP synthesis, versus 62% for the original model. Execution training costs some synthesis ability but does not destroy it.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Mathematics & statistics

Cited by

429 papers of our library cite it (of the 140673 whose bibliographies we read); 62 of them have a page here, 367 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 2112.00114

@misc{anon2021show,
  title = {{Show your work: Scratchpads for intermediate computation with language models}},
  year = {2021},
  eprint = {2112.00114},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2112.00114}
}

References

30 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. Learning to represent programs with graphs. In International Conference on Learning Representations (ICLR), February 2018. find
  2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. find our card
  3. Andrea Banino, Jan Balaguer, and Charles Blundell. Pondernet: Learning to ponder. In 8th ICML Workshop on Automated Machine Learning (AutoML), 2021. find
  4. Emily M. Bender and Alexander Koller. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5185–5198, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v… find our card
  5. David Bieber, Charles Sutton, Hugo Larochelle, and Daniel Tarlow. Learning to execute programs with instruction pointer attention graph neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 8626–8… find
  6. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… find
  7. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde, Jared Kaplan, Harri Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea… find our card
  8. Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. July 2018. find
  9. Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. Robustfill: Neural program learning under noisy I/O. CoRR, abs/1703.07469, 2017. URL http://arxiv.org/abs/1703.07469. ↗
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019. find
  11. Alex Graves. Adaptive computation time for recurrent neural networks. arXiv:1603.08983, 2016. find
  12. Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, abs/1410.5401, 2014. find
  13. Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gomez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield,… find
  14. Lukasz Kaiser and Ilya Sutskever. Neural gpus learn algorithms. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016. find
  15. Karol Kurach, Marcin Andrychowicz, and Ilya Sutskever. Neural random-access machines. In International Conference on Learning Representations, (ICLR), 2016. find
  16. Tom Kwiatkowksi, Luke Zettlemoyer, Sharon Goldwater, and Mark Steedman. Inducing probabilistic CCG grammars from logical form with higher-order unification. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1223–1233, October 2010. find
  17. Belinda Z. Li, Maxwell Nye, and Jacob Andreas. Implicit representations of meaning in neural language models. ArXiv, abs/2106.00737, 2021. find
  18. Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Pretrained transformers as universal computation engines. March 2021. find
  19. Ruchir Puri, David S Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladmir Zolotov, Julian Dolby, Jie Chen, Mihir Choudhury, Lindsey Decker, Veronika Thost, Luca Buratti, Saurabh Pujar, and Ulrich Finkler. Project CodeNet: A Large-Scale AI for code dataset for learning a diversity of coding ta… find
  20. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR, abs/1910.10683, 2019. URL http://arxiv.org/abs/1910.10683. ↗
  21. Scott Reed and Nando de Freitas. Neural programmer-interpreters. In International Conference on Learning Representations (ICLR), 2016. URL http://arxiv.org/pdf/1511.06279v3. ↗
  22. David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical reasoning abilities of neural models. CoRR, abs/1904.01557, 2019. URL http://arxiv.org/abs/1904.01557. arXiv preprint Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, L… ↗ our card
  23. Petar Velickovic and Charles Blundell. Neural algorithmic reasoning. CoRR, abs/2105.02761, 2021. URL https://arxiv.org/abs/2105.02761. ↗
  24. Petar Veličković, Lars Buesing, Matthew C. Overlan, Razvan Pascanu, Oriol Vinyals, and Charles Blundell. Pointer graph networks, 2020a. find
  25. Petar Veličković, Rex Ying, Matilde Padovano, Raia Hadsell, and Charles Blundell. Neural execution of graph algorithms, 2020b. find
  26. Yu Wang, Fengjuan Gao, Linzhang Wang, and Ke Wang. Learning semantic program embeddings with graph interval neural network, 2020. find
  27. Yuk Wah Wong and Raymond J Mooney. Learning for semantic parsing with statistical machine translation. In Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics -, Morristown, NJ, USA, 2006. Association… find
  28. Wojciech Zaremba and Ilya Sutskever. Learning to execute. ArXiv, abs/1410.4615, 2014. find
  29. J Zelle and R Mooney. Learning to parse database queries using inductive logic programming. In National Conference on Artificial Intelligence (AAAI), 1996. find
  30. Luke S Zettlemoyer and Michael Collins. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Uncertainty in Artificial Intelligence, July 2005. find