Elamur Research⌕
☰
home / papers / 2024 / Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process

Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process

Paper · 2024 · Tian Ye, Zicheng Xu, Yuanzhi Li et al.

Summary

Do language models actually reason through grade-school math, or do they reuse memorized templates? Internet-pretrained models cannot answer this because of possible benchmark contamination, so the authors built iGSM, a synthetic generator of GSM8K-style word problems. Each problem comes from a hierarchical "world" of categories (schools, classrooms, backpacks) and a dependency graph. The graph encodes direct, instance and implicit (abstract) relations between quantities. Arithmetic is limited to integers mod 23 and common-sense cues are removed, so only logical reasoning remains. A GPT2-style model with rotary embeddings (RoPE) is trained from scratch on this data, and its hidden states are analysed with six probing tasks. The model reaches about 99% in-distribution accuracy and handles longer reasoning chains than any seen in training. It usually writes shortest solutions, which suggests it plans which parameters it needs before generating. Probes show it also computes all-pair parameter dependencies, a skill its training data never requires, and that many of its errors can be predicted from its internal states. The work offers a controlled way to study reasoning mechanisms and the effect of model shape.

Key points

  • iGSM yields more than 90 trillion distinct solution templates, which rules out memorization by a ~100M-parameter GPT2-small. The GSM8K training set, by contrast, has only 7.5K problems.
  • The world knowledge is drawn from 4 fixed category hierarchies, where every hierarchy spans 4 levels and each level contains 100 items.
  • Probing shows the model mentally identifies the necessary parameters before writing any solution. It also learns dependency relations it never needs for its training data, which the authors present as the first evidence of a model acquiring skills beyond fitting its pretraining data.
  • Typical errors include computing unneeded parameters or parameters not yet computable. GPT-4/GPT-4o show similar failures on iGSM-med pq.
  • Depth matters more than size: a 16-layer, 576-dim transformer solves longer reasoning problems than a 4-layer, 1920-dim model twice its size, even with Chain-of-Thought.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States492 works · 440,462 citations🎓Carnegie Mellon UniversityPittsburgh · United States598 works · 191,225 citations🏢MetaMenlo Park · United States214 works · 241,492 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates138 works · 6,590 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksReasoning · 1,685 works

Cite

Cite as Tian Ye, Zicheng Xu, Yuanzhi Li and Zeyuan Allen-Zhu. 2024. Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process. arXiv:2407.20311.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2407.20311 · arXiv 2407.20311

@misc{ye2024physics,
  title = {{Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process}},
  author = {Tian Ye, Zicheng Xu, Yuanzhi Li and Zeyuan Allen-Zhu},
  year = {2024},
  doi = {10.48550/arxiv.2407.20311},
  eprint = {2407.20311},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2407.20311}
}

References

23 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 1, Learning Hierarchical Language Structures. ArXiv e-prints, abs/2305.13673, May 2023. Full version available at http://arxiv.org/abs/2305.13673. ↗
  2. Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 3.2, Knowledge Manipulation. ArXiv e-prints, abs/2309.14402, September 2023. Full version available at http://arxiv.org/abs/2309.14402. ↗
  3. Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 3.1, Knowledge Storage and Extraction. In ICML, 2024. Full version available at http://arxiv.org/abs/2309.14316. ↗
  4. Zeyuan Allen-Zhu and Yuanzhi Li. Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws. ArXiv e-prints, abs/2404.05405, April 2024. Full version available at http://arxiv.org/abs/2404.05405. ↗
  5. Zeyuan Allen-Zhu, Yuanzhi Li, and Zhao Song. A convergence theory for deep learning via overparameterization. In ICML, 2019. Full version available at http://arxiv.org/abs/1811.03962. ↗
  6. Cem Anil, Yuhuai Wu, Anders Andreassen, Aitor Lewkowycz, Vedant Misra, Vinay Ramasesh, Ambrose Slone, Guy Gur-Ari, Ethan Dyer, and Behnam Neyshabur. Exploring length generalization in large language models. Advances in Neural Information Processing Systems, 35:38546–38556, 2022.
  7. Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. GPT-NeoX-20B: An open-source autoregressive… — GPT-NeoX-20B: An Open-Source Autoregressive Language Model
  8. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
  9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
  10. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. find
  11. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. Textbooks are all you need. arXiv preprint arXiv:2306.11644, 2023. find
  12. John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4129–4138,…
  13. Samy Jelassi, Stéphane d’Ascoli, Carles Domingo-Enrich, Yuhuai Wu, Yuanzhi Li, and François Charton. Length generalization in arithmetic transformers. arXiv preprint arXiv:2306.15400, 2023. find
  14. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
  15. Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint arXiv:2309.05463, 2023.
  16. Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, and Yi Zhang. TinyGSM: achieving \> 80% on GSM8k with small language models. arXiv preprint arXiv:2312.09241, 2023. find
  17. OpenAI. Gpt-4 technical report, 2023.
  18. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. find
  19. Lance J Rips. The psychology of proof: Deductive reasoning in human thinking. Mit Press, 1994. find
  20. Jianlin Su, Yu Lu, Shengfeng Pan, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, 2021.
  21. Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems. arXiv preprint arXiv:xxxx.xxxxx, 2024. to appear. find
  22. Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Dylan Slack, Qin Lyu, et al. A careful examination of large language model performance on grade school arithmetic. arXiv preprint arXiv:2405.00332, 2024.
  23. Hattie Zhou, Arwen Bradley, Etai Littwin, Noam Razin, Omid Saremi, Josh Susskind, Samy Bengio, and Preetum Nakkiran. What algorithms can transformers learn? a study in length generalization. arXiv preprint arXiv:2310.16028, 2023.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ye-2024-physics-language-models-part-2/