Elamur Research⌕
☰
home / papers / 2022 / Language Model Cascades

Language Model Cascades

Paper · 2022 · David Dohan, Winnie Xu, Aitor Lewkowycz et al.

Summary

The paper addresses how to reason about the many ad hoc ways large language models are chained or repeatedly queried. Examples include step-by-step reasoning prompts, answer checkers, and calls to external tools. It is a position paper presented at an ICML 2022 workshop, Beyond Bayes. The authors propose treating any such composition as a probabilistic program whose random variables hold strings. Each conditional distribution is supplied by a pretrained LM, and observed strings can be used for conditioning to infer unknown ones. They call these programs "language model cascades" and build a small trace-based PPL embedded in Python. Within this framework they re-express several existing methods as graphical models or programs: scratchpads and chain of thought, semi-supervised thought imputation, STaR, Selection-Inference, verifiers, and tool use. As a small experiment, a cascade playing BIG-bench Twenty Questions solves 29% of tasks. The value of the work is mainly conceptual. A shared formalism could allow generic inference, tuning and prompt selection to be applied across many reasoning pipelines.

Key points

  • The implementation uses effect handlers (in the spirit of Pyro) and coroutines. It supports arbitrary control flow and recursion, and the code is published at model-cascades.github.io.
  • STaR is reinterpreted as a stochastic EM-like procedure. In the E step, missing rationales are filled in by rejection sampling or by conditioning on the known answer. The M step then fine-tunes on these imputed rationales.
  • In the Twenty Questions experiment, LaMDA 137B played both Alice and Bob. The setup sampled 50 conversations per concept at temperature 1.0, discarded malformed chains early, and masked the concept name in Alice's replies.
  • The authors acknowledge that only rejection or forward sampling was evaluated. Efficient inference over string variables remains the core open problem.
  • Proposed future directions include:
    • using LMs as learned proposal or guide distributions;
    • extending cascades to multimodal settings;
    • searching for cascade structures automatically via program induction.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States492 works · 440,462 citations🏢AlphabetMountain View · United States4 works · 3,417 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-dickstein, Kevin Murphy and Charles Sutton. 2022. Language Model Cascades. arXiv:2207.10342.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2207.10342 · arXiv 2207.10342

@misc{dohan2022language,
  title = {{Language Model Cascades}},
  author = {David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A. Saurous, Jascha Sohl-dickstein, Kevin Murphy and Charles Sutton},
  year = {2022},
  doi = {10.48550/arxiv.2207.10342},
  eprint = {2207.10342},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2207.10342}
}

References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Goodman, N. D., Tenenbaum, J. B., and Gerstenberg, T. Concepts in a probabilistic language of thought. (010), 06/2014 2014. find
  2. ARC’s first technical report: Eliciting latent Kochurov, M., Carroll, C., Wiecki, T., and Lao, J. PyMC4: knowledge - AI alignment forum, 2022. URL Exploiting coroutines for implementing a probabilistic https://www.alignmentforum.org/posts/qHCDysDnvhteW7kRd/arc-s-first-technical-report-eliciting-late… ↗
  3. Lake, B. M., Salakhutdinov, R., and Tenenbaum, J. B. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332–1338, December 2015. find
  4. Levine, S. Reinforcement learning and control as probabilistic inference: Tutorial and review. May 2018. find
  5. Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., Jiang, X., Cobbe, K., Eloundou, T., Krueger, G., Button, K., Knight, M., Chess, B., and Schulman, J. WebGPT: Browser-assisted question-answering with human feedback. December 2021.
  6. Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., Sutton, C., and Odena, A. Show your work: Scratchpads for intermediate computation with language models. November 2021.
  7. Ortega, P. A., Kunesch, M., Delétang, G., Genewein, T., Grau-Moya, J., Veness, J., Buchli, J., Degrave, J., Piot, B., Perolat, J., et al. Shaking the foundations: delusions in sequence models for interaction and control. arXiv preprint arXiv:2110.10819, 2021. — [arXiv:2110.10819](https://arxiv.org/abs/2110.10819) find
  8. Phan, D., Pradhan, N., and Jankowiak, M. Composable effects for flexible and accelerated probabilistic programming in NumPyro. December 2019. find
  9. Polu, S. and Sutskever, I. Generative language modeling for automated theorem proving. September 2020.
  10. Saunders, W., Yeh, C., Wu, J., Bills, S., Ouyang, L., Ward, J., and Leike, J. Self-critiquing models for assisting human evaluators, 2022. URL https://arxiv.org/abs/2206.05802. ↗
  11. Scheurer, J., Campos, J. A., Chan, J. S., Chen, A., Cho, K., and Perez, E. Training language models with language feedback. April 2022. find
  12. Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., Li, Y., Lee, H., Zheng, H. S., Ghafouri, A., Menegali, M., Huang, Y., Krikun, M., Lepikhin, D., Qin, J., Chen, D., Xu, Y., Chen, Z., Roberts, A., Bosma, M., Zhao, V., Zhou, Y.,… — LaMDA: Language Models for Dialog Applications
  13. Tolpin, D., van de Meent, J., Yang, H., and Wood, F. D. Design and implementation of probabilistic programming language anglican. CoRR, abs/1608.05263, 2016. URL http://arxiv.org/abs/1608.05263. van de Meent, J.-W., Paige, B., Yang, H., and Wood, F. An introduction to probabilistic programming. — [arXiv:1608.05263](https://arxiv.org/abs/1608.05263) ↗
  14. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. Self-Consistency improves chain of thought reasoning in language models. March 2022.
  15. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D. Chain of thought prompting elicits reasoning in large language models. January 2022. — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  16. Wu, M. and Goodman, N. Foundation posteriors for approximate probabilistic inference. May 2022. find
  17. Wu, T., Jiang, E., Donsbach, A., Gray, J., Molina, A., Terry, M., and Cai, C. J. Promptchainer: Chaining large language model prompts through visual programming, 2022. URL https://arxiv.org/abs/2203.06566. — [arXiv:2203.06566](https://arxiv.org/abs/2203.06566) ↗
  18. Zelikman, E., Wu, Y., and Goodman, N. D. Star: Bootstrapping reasoning with reasoning. arXiv preprint arXiv:2203.14465, 2022.
  19. Zeng, A., Wong, A., Welker, S., Choromanski, K., Tombari, F., Purohit, A., Ryoo, M., Sindhwani, V., Lee, J., Vanhoucke, V., and Florence, P. Socratic models: Composing Zero-Shot multimodal reasoning with language. April 2022.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dohan-2022-language-model-cascades/