Arithmetic with Language Models: from Memorization to Computation
Paper · 2023 · Davide Maltoni and Matteo Ferrara
Summary
The paper asks whether a language model trained on next-token prediction can genuinely compute arithmetic results, rather than recalling or smoothly interpolating seen examples. The authors chose binary addition and multiplication as a controlled testbed. These tasks use a tiny vocabulary, and a single flipped input bit can change many output bits, so nearest-neighbour style interpolation fails on new inputs. They trained small, non-pretrained Transformers from scratch on 7-bit operands: the original encoder-decoder model and the decoder-only nanoGPT. Both models learned the tasks. The authors then removed chosen input regions from training to test extrapolation, analysed correlations in internal embeddings, and applied amnesic probing. The evidence points to an Encoding-Regression-Decoding (ERD) mechanism. Input tokens are first mapped into an internal numeric "value" representation, the computation happens as a regression in that space, and the result is decoded back into output tokens. The work helps explain how LMs might reach arithmetic generalization beyond memorization.
Key points
- Task format: operands range over 0–127 and are written as 15-token prompts (two 7-bit strings plus an operator). The vocabulary has about 5 tokens, and outputs are 8 bits for addition and 14 bits for multiplication. Both inputs and outputs are written least-significant bit first.
- Bit order: reverse (little-endian) output order made training much faster and more stable than plain order. Input order made no difference.
- nanoGPT setup and accuracy: the model has 298K parameters, d_model 64, 8 heads and 6 layers. On the VS_t held-out subset it reached 100% for addition and 99.9% for multiplication. On VS_v it dropped to 82.0% and 80.6%.
- Amnesic probing setup: a linear regressor recovered the sum from decoder layer-3 embeddings with an rmse of 0.28.
- Amnesic probing result: removing that value information cut addition sequence accuracy from 100% to 0.13%. Removing the same amount of information along random directions left accuracy at 100%.
Institutions
ItalyFields, topics and concepts
📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works
🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 4,145 works
Cite
Cite as Davide Maltoni and Matteo Ferrara. 2023. Arithmetic with Language Models: from Memorization to Computation. arXiv:2308.01154 · \Neural Networks, vol. 179, 2024.
@misc{maltoni2023arithmetic,
title = {{Arithmetic with Language Models: from Memorization to Computation}},
author = {Davide Maltoni and Matteo Ferrara},
year = {2023},
doi = {10.1016/j.neunet.2024.106550},
eprint = {2308.01154},
archivePrefix = {arXiv},
url = {https://doi.org/10.1016/j.neunet.2024.106550}
}References
- Belinkov Y., Probing Classifiers: Promises, Shortcomings, and Advances, Computational Linguistics, vol. 48, pp. 207-219, 2022. Bubeck S., Chandrasekaran V., Eldan R., Gehrke J., Horvitz E., Kamar E., Lee P., Lee Y., Li Y., Lundberg S., Nori H., Palangi H., Ribeiro M., Zhang Y., Sparks of Artificial…
- Räuker T., Ho A., Casper S., Hadfield-Menell D., Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks, arXiv: 2207.13243, 2023. Ravfogel S., Elazar Y., Gonen H., Twiton M., Goldberg Y., Null It Out: Guarding Protected Attributes by Iterative Nullspace Project…
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/maltoni-2023-arithmetic-language-models-memorization-computation/