Elamur Research⌕
☰
home / papers / 2023 / GPT-4 Can't Reason

GPT-4 Can't Reason

Paper · 2023 · Konstantine Arkoudas

Summary

This position paper by Konstantine Arkoudas asks whether GPT-4, then considered the strongest LLM, can actually reason. The author argues that standard reasoning benchmarks are poor evidence. Training data may be contaminated in ways that substring checks miss. Most benchmarks are also multiple-choice or binary classification tasks, so a model can score well by exploiting spurious statistical cues. Instead of using a corpus, he builds a set of 21 elementary problems, most written from scratch and some, such as Wason's selection task, altered to hide their origin. He then examines GPT-4's answers, explanations and proof attempts by hand, including how it reacts in dialogue when its mistakes are pointed out. The author's conclusion is that GPT-4 solves none of the problems and shows deep confusion about basic logical concepts. He rejects both a weak claim (average-human reasoning competence) and a strong claim (reliability sufficient for science, medicine or engineering). The paper matters because it pushes the field toward generative, manually scrutinized evaluation of LLM reasoning.

Key points

  • The evaluation uses 21 diverse, mostly original reasoning problems, framed around a liberal notion of deductive reasoning and analysed qualitatively rather than with numeric metrics.
  • The author reports that GPT-4 fails all 21 problems. Its errors include being unable to tell left from right (Section 1.3.12), and he considers its flawed explanations more worrying than its wrong final answers.
  • Automatic metrics such as ROUGE-L, BLEU and embedding-based similarity are judged unfit for assessing natural-language derivations.
  • The paper critiques Liu et al.'s GPT-4 study, which relied on LogiQA, ReClor, ConTRoL, MED, ConjNLI, TaxiNLI and AR-LSAT. All of these are multiple-choice or classification tasks and may resemble pre-2021 data.
  • On theoretical grounds, the author notes that general first-order reasoning is undecidable, so no LLM can solve arbitrary reasoning problems soundly and completely. He nonetheless treats the question for any specific system as an empirical one.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Konstantine Arkoudas. 2023. GPT-4 Can't Reason. arXiv:2308.03762.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2308.03762 · arXiv 2308.03762

@misc{arkoudas2023gpt,
  title = {{GPT-4 Can't Reason}},
  author = {Konstantine Arkoudas},
  year = {2023},
  doi = {10.48550/arxiv.2308.03762},
  eprint = {2308.03762},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.03762}
}

References

11 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Arkoudas, K. and Musser, D., Fundamental Proof Methods in Computer Science, MIT Press, 2017. find
  2. Barwise, J. and Perry, J., Situations and Attitudes, MIT Press, 1983. find
  3. Karpas, E., Abend, O., Belinkov, Y., Lenz, B., Lieber, O., Ratner, N., . . ., Tenenholtz, M., MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning, 2022. find
  4. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y., ReAct: Synergizing Reasoning and Acting in Language Models, https://arxiv.org/abs/2210.03629 , 2023. ↗
  5. Planken, L., Temporal Reasoning Problems and Algorithms for Solving Them: Literature Survey, 2008. find
  6. McCoy, T., Pavlick, E., Linzen, T., Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
  7. Liu H., Ning R., Teng, Z., Liu, J., Zhou, Q., Zhang, Y., Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4, 2023.
  8. OpenAI, GPT-4 Technical Report, 2023.
  9. Wang, J., Hu, X., Hou, W., Chen, H., Zheng, R., Wang, Y., . . ., Xie, X., On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective, 2023. find
  10. Niven, T., Kao, H.-Y. Probing Neural Network Comprehension of Natural Language Arguments, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. find
  11. Johnson-Laird, P.N., How We Reason, Oxford University Press, 2006. find

Updated

This page in Markdown → https://md.research.elamur.ai/free/en/papers/arkoudas-2023-gpt-4-t-reason/