Elamur Research⌕
☰
home / papers / 2023 / Why think step by step? Reasoning emerges from the locality of experience

Why think step by step? Reasoning emerges from the locality of experience

Paper · 2023 · Ben Prystawski, Michael Y. Li and Noah D. Goodman

Summary

Chain-of-thought prompting often makes large language models answer better, yet it adds no new information. The authors ask why. Their hypothesis is that step-by-step reasoning pays off when training data comes in overlapping local clusters of strongly related variables. A model can then link reliable nearby inferences to estimate relationships between variables it never saw together. They first prove this in a simple setting, where a density estimator that predicts autoregressively learns from locally observed samples drawn from a chain-structured probabilistic model. Reasoning through intermediate variables lowers bias, which they call a "reasoning gap." They then train autoregressive transformers from scratch on samples from Bayes nets, where each sample shows only a subset of variables. They compare how well the models match true conditional probabilities with and without generated intermediate steps. Intermediate steps help only when the training data's locality follows the dependency structure of the net. Local data combined with self-generated reasoning is also far more data-efficient than training on all variables. The work offers a statistical account of why chain-of-thought works and points to how training corpora could be curated.

Key points

  • The paper is arXiv:2304.03843 by Prystawski, Li and Goodman of Stanford, presented at NeurIPS 2023.
  • Training sequences are lists of variable index/value pairs. An observation distribution p_obs decides which variable subsets appear together. Conditions include fully observed, local (geom) and wrong-locality data.
  • Three estimators are compared, scored by mean squared error against true conditional probabilities:
    • direct prediction, which answers immediately;
    • scaffolded generation, which first generates ideal d-separating intermediate variables and marginalizes over them with Monte Carlo samples;
    • free generation, where the model chooses its own intermediate variables.
  • Free generation helps about as much as ideal scaffolds. It brings no gain over direct prediction when training data includes all variables. With wrong locality, model estimates drift toward the targets' marginal probabilities.
  • Stated limitations:
    • the findings fit zero-shot chain-of-thought best, not few-shot reasoning examples;
    • the settings are simple propositional worlds;
    • the theory does not explain why transformers revert to marginals.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Psychology · 1,673 worksExperimental and Cognitive Psychology · 950 worksNeuroscience · 192 worksCognitive Neuroscience · 191 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Ben Prystawski, Michael Y. Li and Noah D. Goodman. 2023. Why think step by step? Reasoning emerges from the locality of experience. arXiv:2304.03843.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2304.03843 · arXiv 2304.03843

@misc{prystawski2023why,
  title = {{Why think step by step? Reasoning emerges from the locality of experience}},
  author = {Ben Prystawski, Michael Y. Li and Noah D. Goodman},
  year = {2023},
  doi = {10.48550/arxiv.2304.03843},
  eprint = {2304.03843},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2304.03843}
}

References

23 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. N. Shepard, “The step to rationality: The efficacy of thought experiments in science, ethics, and free will,” Cognitive Science, vol. 32, no. 1, pp. 3–35, 2008. find
  2. D. C. Dennett, Intuition pumps and other tools for thinking. WW Norton & Company, 2013. find
  3. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020.
  4. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al., “Palm: Scaling language modeling with pathways,” arXiv preprint arXiv:2204.02311, 2022.
  5. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023.
  6. B. bench authors, “Beyond the imitation game: Quantifying and extrapolating the capabilities of language models,” Transactions on Machine Learning Research, 2023.
  7. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al., “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168, 2021.
  8. M. Nye, A. J. Andreassen, G. Gur-Ari, H. Michalewski, J. Austin, D. Bieber, D. Dohan, A. Lewkowycz, M. Bosma, D. Luan, et al., “Show your work: Scratchpads for intermediate computation with language models,” arXiv preprint arXiv:2112.00114, 2021.
  9. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” arXiv preprint arXiv:2205.11916, 2022.
  10. J. Wei, X. Wang, D. Schuurmans, M. Bosma, brian ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems (A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, eds.), 2022. — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  11. D. Dohan, W. Xu, A. Lewkowycz, J. Austin, D. Bieber, R. G. Lopes, Y. Wu, H. Michalewski, R. A. Saurous, J. Sohl-dickstein, et al., “Language model cascades,” arXiv preprint arXiv:2207.10342, 2022.
  12. A. K. Lampinen, I. Dasgupta, S. C. Chan, K. Matthewson, M. H. Tessler, A. Creswell, J. L. McClelland, J. X. Wang, and F. Hill, “Can language models learn from explanations in context?,” arXiv preprint arXiv:2204.02329, 2022.
  13. E. Zelikman, Y. Wu, J. Mu, and N. Goodman, “STaR: Bootstrapping reasoning with reasoning,” in Advances in Neural Information Processing Systems (A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, eds.), 2022.
  14. X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou, “Self-consistency improves chain of thought reasoning in language models,” in The Eleventh International Conference on Learning Representations, 2022.
  15. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le, et al., “Least-to-most prompting enables complex reasoning in large language models,” arXiv preprint arXiv:2205.10625, 2022.
  16. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” arXiv preprint arXiv:2305.10601, 2023. find
  17. D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” Journal of machine Learning research, vol. 3, no. Jan, pp. 993–1022, 2003.
  18. D. M. Blei and J. D. Lafferty, “A correlated topic model of Science,” The Annals of Applied Statistics, vol. 1, no. 1, pp. 17 – 35, 2007. find
  19. S. C. Chan, A. Santoro, A. K. Lampinen, J. X. Wang, A. K. Singh, P. H. Richemond, J. Mc-Clelland, and F. Hill, “Data distributional properties drive emergent in-context learning in transformers,” in Advances in Neural Information Processing Systems (A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, eds… find
  20. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019. find
  21. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, et al., “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38–45,…
  22. R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715–1725, 2016.
  23. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proceedings of the 3rd International Conference on Learning Representations (ICLR), 2015.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/prystawski-2023-why-think-step-step-reasoning/