Faith and Fate: Limits of Transformers on Compositionality (2023 · 327 citations)
Large transformer LLMs can handle some demanding reasoning tasks yet fail on simple procedural ones. The authors ask whether these failures are accidental or reflect deeper structural limits. They examine three compositional tasks: long-form multiplication with operands of up to 5 digits, Einstein-style puzzles that…
PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) Planning (2023 · 25 citations)
The paper asks whether compact language models can handle procedural planning, meaning turning a high-level goal into an ordered list of steps.

All works

3 works of theirs in our library, the most cited first; 2 with a page here, the others link their original until we summarise them.

Scaling Language Models: Methods, Analysis & Insights from Training Gopher ↗arXiv:2112.11446 · no summary yetJack W. Rae et al.2021 · 928 citations
Faith and Fate: Limits of Transformers on CompositionalityConference on Neural Information Processing Systems (NeurIPS)Nouha Dziri, Ximing Lu, Melanie Sclar, et al.2023 · 327 citations
PlaSma: Making Small Language Models Better Procedural Knowledge Models for (Counterfactual) PlanningarXiv:2305.19472Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, et al.2023 · 25 citations

Co-authors here

12 of 16 co-authors with a page, the most papers together first.

Where their works here were published, the most works first, with the editions of those that have a page.

Papers of our library cite their works 937 times (of the 149,439 papers whose bibliographies we read); 244 citing papers with a page here →.

Updated 30 October 2026