Elamur Research⌕
☰
home / papers / 2026 / Can Large Language Models Generalize Analogy Solving Like Children Can?

Can Large Language Models Generalize Analogy Solving Like Children Can?

Paper · 2026 · Claire E. Stevenson, Alexandra Pafford, Han L. J. van der Maas et al.

Summary

This study tests whether large language models can carry analogical reasoning into unfamiliar domains as flexibly as people, including young children. The authors used letter-string analogies (e.g., a b → a c, so j k → ?). These need little background knowledge, which makes them suitable for both 7–9-year-olds and LLMs. Each of five base items appeared in three versions: the familiar Latin alphabet, the Greek alphabet as near transfer, and an invented ordered list of keyboard symbols as far transfer. The transformations were successor, second successor, predecessor and repetition. Children, adults and four LLMs solved the same 15 problems with identical prompts. Both human groups kept their performance across all three alphabets. The models matched or beat children on Latin items but declined on Greek items and dropped steeply on Symbol items. Follow-up tasks indicate that the models struggle to build an ordered representation of a new "alphabet" on the fly. This suggests that strong results on familiar material can overstate how robust LLM analogical reasoning really is.

Key points

  • Participants were 42 children (7–9 years, recruited at a Dutch Montessori school), 62 adults (via Prolific) and 55 runs each of Claude-3.5, Gemma-2 27B, GPT-4o and Llama-3.1 405B. The models ran at temperature 0 with a 10-token output limit.
  • Smaller models from the same families (GPT-3/3.5/4, Claude-3, Gemma-2-9B, Llama-3.1-8B/70B) were also tested. Performance generally rose with model size, most clearly in the Symbol domain.
  • In a separate Rule Check task, models applied repetition rules to new alphabets without trouble. Predecessor and second-successor rules were much harder for them, since these depend on knowing the alphabet's order.
  • Models sometimes gave "literal" answers (e.g., replacing a letter with the exact character from the example) in roughly 5–21% of cases, while humans never did. Children's wrong answers were further from the target by Levenshtein distance; LLM errors stayed just under 2 edits.
  • Reordering the Symbol list by Unicode value helped only Claude and Gemma somewhat, and the overall human–LLM gap remained.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of AmsterdamAmsterdam · Netherlands144 works · 173,906 citations🔬Santa Fe InstituteSanta Fe · United States7 works · 646 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Claire E. Stevenson, Alexandra Pafford, Han L. J. van der Maas and Melanie Mitchell. 2026. Can Large Language Models Generalize Analogy Solving Like Children Can?. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.614

@article{stevenson2026can,
  title = {{Can Large Language Models Generalize Analogy Solving Like Children Can?}},
  author = {Claire E. Stevenson, Alexandra Pafford, Han L. J. {van der Maas} and Melanie Mitchell},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.614},
  url = {https://doi.org/10.1162/tacl.a.614}
}

References

44 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. being given the ordered list of letters/symbols before each item. This inability was clear for both larger and smaller LLMs, although relative performance did scale with model size. A possible explanation lies in work studying the internal representations of LLMs, where abstract concepts like ‘‘anto… find
  2. Indeed, Webb et al. (2024) found that GPT-4 can only perform these abstractions by creating and executing code to map the novel alphabet to new positions and compute previous and next letters. This is of course very different from how children solve such problems. In contrast, our results show that… find
  3. Similarly, Doumas et al. (2022) argue that robust analogical transfer is a uniquely human ability. Based on our findings so far we concur, and now ask the question: Is generalization to unfamiliar domains indeed what separates human general intelligence from that of artificial general intelligence?… find
  4. Anthropic. 2024. Introducing the next generation of claude. Anthropic Preprint. find
  5. Susan M. Barnett and Stephen J. Ceci. 2002. When and where do we apply what we learn?: A taxonomy for far transfer. Psychological Bulletin, 128(4):612. https://doi.org/10.1037/0033-2909.128.4.612, PubMed: 12081085 ↗
  6. Katarzyna Bobrowicz, Felicia Lindström, Marcus Lindblom Lovén, and Elia Psouni. 2020. Flexibility in problem solving: Analogical transfer of tool use in toddlers is immune to delay. Frontiers in Psychology, 11:573730. https://doi.org/10.3389/fpsyg .2020.573730, PubMed: 33123052 — [doi:10.3389/fpsyg](https://doi.org/10.3389/fpsyg) ↗
  7. Zhe Chen. 1996. Children’s analogical problem solving: The effects of superficial, structural, and procedural similarity. Journal of Experimental Child Psychology, 62(3):410–431. https://doi.org/10.1006/jecp .1996.0037, PubMed: 8691121 — [doi:10.1006/jecp](https://doi.org/10.1006/jecp) ↗
  8. Leonidas A. A. Doumas, Guillermo Puebla, Andrea E. Martin, and John E. Hummel. 2022. A theory of relation learning and cross-domain generalization. Psychological Review. https://doi.org/10.1037/rev0000346, PubMed: 35113620 ↗
  9. Gemma Team. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. find
  10. Gael Gendron, Qiming Bao, Michael Witbrock, and Gillian Dobbie. 2024. Large language models are not strong abstract reasoners. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence, IJCAI-24. International Joint Conferences on Artificial Intelligence Organization. \#\# A…
  11. Dedre Gentner and Christian Hoyos. 2017. Analogy and abstraction. Topics in Cognitive Science, 9(3):672–693. https://doi.org/10.1111/tops.12278, PubMed: 28621480 — [doi:10.1111/tops.12278](https://doi.org/10.1111/tops.12278) ↗
  12. Dedre Gentner and Cecile Toupin. 1986. Systematicity and surface similarity in the development of analogy. Cognitive Science, 10(3):277–300. https://doi.org /10.1016/S0364-0213(86)80019-2 doi.org/10.3758/s13428-019-01312-3, PubMed: 31898298 — [doi:10.1016/s0364-0213(86)80019-2](https://doi.org/10.1016/s0364-0213%2886%2980019-2) ↗
  13. Tamar Johnson, Mathilde ter Veen, Rochelle Choenni, Han van der Maas, Ekaterina Shutova, and Claire E. Stevenson. 2025. Do large language models solve verbal analogies like children do? In Proceedings of the 29th Conference on Computational Natural Language Learning, pages 627–639. https://doi.org/1… find
  14. Mariel K. Goddu, Tania Lombrozo, and Alison Gopnik. 2020. Transformations and transfer: Preschool children understand abstract relations and reason analogically in a causal task. Child Development, 91(6):1898–1915. https://doi.org/10.1111/cdev .13412, PubMed: 32880903 — [doi:10.1111/cdev](https://doi.org/10.1111/cdev) ↗
  15. Usha Goswami. 1991. Analogical reasoning: What develops? A review of research and theory. Child Development, 62(1):1–22. https://doi.org/10.1111/j.1467-8624 .1991.tb01511.x Damian Hodel and Jevin West. 2023. Response to ‘‘Emergent analogical reasoning in large language models’’. arXiv preprint arXiv… ↗
  16. Laura L. Jones, Matt J. Kmiecik, John L. Irwin, and Robert G. Morrison. 2022. Differential effects of semantic distance, distractor salience, and relations in verbal analogy. Psychonomic Bulletin & Review. https://doi.org/10 .3758/s13423-022-02062-8, PubMed: 35132581 find
  17. Martha Lewis and Melanie Mitchell. 2025. Evaluating the robustness of analogical reasoning in large language models. Transactions on Machine Learning Research. https://doi.org/10.1007/978-3-031-76646-6 4 — [doi:10.1007/978-3-031-76646-6](https://doi.org/10.1007/978-3-031-76646-6) ↗
  18. Douglas R. Hofstadter. 1984. The Copycat project: An experiment in nondeterminism and creative analogies, Massachusetts Institute of Technology. find
  19. R. Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L. Griffiths. 2024. Embers of autoregression show how large language models are shaped by the problem they are trained to solve. Proceedings of the National Academy of Sciences, 121(41):e2322420121. Edited by Richard Shiffrin. http… — [doi:10.1073/pnas.2322420121](https://doi.org/10.1073/pnas.2322420121) find
  20. Douglas R. Hofstadter and Melanie Mitchell. 1994. The copycat project: A model of mental fluidity and analogy-making. In Advances in Connectionist and Neural Computation Theory, volume 2, pages 31–112. Ablex, Norwood, NJ. find
  21. Keith J. Holyoak. 2012. Analogy and relational reasoning. In Keith J. Holyoak and Robert G. Morrison, editors, The Oxford Handbook of Thinking and Reasoning, pages 234–259. https://doi.org/10.1093/oxfordhb/9780199734689.013.0013 — [doi:10.1093/oxfordhb](https://doi.org/10.1093/oxfordhb) ↗
  22. Melanie Mitchell. 2021. Abstraction and analogy-making in artificial intelligence. Annals of the New York Academy of Sciences, 1505(1):79–101. https://doi.org/10 .1111/nyas.14619, PubMed: 34173249
  23. Arsenii Kirillovich Moskvichev, Victor Vikram Odouard, and Melanie Mitchell. 2023. The conceptARC benchmark: Evaluating understanding and generalization in the ARC domain. Transactions on Machine Learning Research. find
  24. Keith J. Holyoak, Ellen N. Junn, and Dorrit O. Billman. 1984. Development of analogical problem-solving skill. Child Development 2042–2055. https://doi.org/10.2307/1129778, PubMed: 6525888 ↗
  25. Timothy M. Mulholland, James W. Pellegrino, and Robert Glaser. 1980. Components of geometric analogy solution. Cognitive Psychology, 12(2):252–284. https://doi.org/10 .1016/0010-0285(80)90011-0, PubMed: 7371379 find
  26. Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. CoRR, abs/2212.10403.
  27. Nicholas Ichien, Hongjing Lu, and Keith J. Holyoak. 2020. Verbal analogy problem sets: An inventory of testing materials. Behavior Research Methods, 52(5):1803–1816. https:// find
  28. OpenAI. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774.
  29. Gustaw Opiełka, Hannes Rosenbusch, and Claire E. Stevenson. 2025. Analogical reasoning inside large language models: Concept vectors and the limits of abstraction. arXiv preprint arXiv:2503.03666. the what and when of analogical reasoning across analogy formats: An eye-tracking and machine learning… — [doi:10.1111/cogs.13208](https://doi.org/10.1111/cogs.13208) find
  30. Gustaw Opiełka, Hannes Rosenbusch, Veerle Vijverberg, and Claire E. Stevenson. 2024. Do large language models solve ARC visual analogies like people do? In Proceedings of the 46th Annual Conference of the Cognitive Science Society. find
  31. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — Llama 2: Open Foundation and Fine-Tuned Chat Models
  32. Yasaman Razeghi, Robert L. Logan IV, Matt Gardner, and Sameer Singh. 2022. Impact of pretraining term frequencies on few-shot reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 840–854. https://doi.org/10.18653/v1/2022.findings -emnlp.59 — [doi:10.18653/v1/2022.findings](https://doi.org/10.18653/v1/2022.findings) ↗
  33. Lindsey E. Richland, Robert G. Morrison, and Keith J. Holyoak. 2006. Children’s development of analogical reasoning: Insights from scene analogy problems. Journal of Experimental Child Psychology, 94(3):249–273. https://doi.org/10.1016/j.jecp .2006.02.002, PubMed: 16620867 — [doi:10.1016/j.jecp](https://doi.org/10.1016/j.jecp) ↗
  34. Roger C. Schank. 1980. How much intelligence is there in artificial intelligence? Intelligence, 4:1–14. https://doi.org/10 .1016/0160-2896(80)90002-1 find
  35. Richard Shiffrin and Melanie Mitchell. 2023. Probing the psychology of AI models. Proceedings of the National Academy of Sciences, 120(10):e2300963120. https://doi.org/10.1073/pnas.2300963120, PubMed: 36857344 ↗
  36. Taylor Webb, Keith J. Holyoak, and Hongjing Lu. 2023. Emergent analogical reasoning in large language models. Nature Human Behaviour, 7(9):1526–1541. https://doi.org/10 .1038/s41562-023-01659-w, PubMed: 37524930 find
  37. Claire E. Stevenson and Marian Hickendorff. 2018. Learning to solve figural matrix analogies: The paths children take. Learning and Individual Differences, 66:16–28. https://doi.org/10.1016/j.lindif.2018.04.010 — [doi:10.1016/j.lindif.2018.04.010](https://doi.org/10.1016/j.lindif.2018.04.010) ↗
  38. Jean-Pierre Thibaut and Robert M. French. 2016. Analogical reasoning, control and executive functions: A developmental investigation with eye-tracking. Cognitive Development, 38:10–26. https://doi.org/10.1016/j.cogdev.2015.12.002 ↗
  39. Taylor Webb, Keith J. Holyoak, and Hongjing Lu. 2024. Evidence from counterfactual tasks supports emergent analogical reasoning in large language models. arXiv preprint arXiv:2404.13070. https://doi.org/10 .1093/pnasnexus/pgaf135, PubMed: 40432905 — [arXiv:2404.13070](https://arxiv.org/abs/2404.13070) find
  40. Jean-Pierre Thibaut, Yannick Glady, and Robert M. French. 2022. find
  41. Understanding Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent abilities of large language models. Tran…
  42. Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. 2024. Reasoning or reciting? Exploring the capabilities and limitations of language models through counterfactual tasks. In Proceed-Eunice Yiu, Maan Qraitem, Charlie Wong, Anisa No… find
  43. A Open Science Practices Ethics This study was approved by the ethics board at the University of Amsterdam, Social and Behavioural Sciences on May 24, 2023 (ID: FMG-2495). find
  44. Preregistration The hypotheses, methods, and analyses were preregistered on the Open Science Framework (OSF) prior to ethical approval. These were updated on July 26, 2023, to accommodate new methods for LLM data collection. The main deviation from preregistration was using different LLMs than previ… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/stevenson-2026-can-large-language-models-generalize/