Elamur Research⌕
☰
home / papers / 2024 / Premise Order Matters in Reasoning with Large Language Models

Premise Order Matters in Reasoning with Large Language Models

Paper · 2024 · Xinyun Chen, Ryan A. Chi, Xuezhi Wang et al.

Summary

Changing the order of premises does not change a logical or arithmetic problem, but this paper shows that large language models are highly sensitive to it. The authors built two test resources. The first is a synthetic deductive-reasoning benchmark adapted from SimpleLogic. It contains modus ponens problems whose premises are shuffled to set distances from the order of the ground-truth proof, measured with Kendall tau. The second is R-GSM, in which GSM8K word problems have their sentences reordered by hand without altering the answer. They evaluated GPT-4-turbo, GPT-3.5-turbo, PaLM 2-L and Gemini 1.0 Pro using zero-shot greedy decoding. Every model did best when premises appeared in the same sequence as the proof steps. Accuracy could fall by more than 30% under other orderings, and irrelevant rules made the effect worse. The finding suggests that LLMs prefer to reason in a single left-to-right pass rather than jumping around the input. This is a robustness issue that matters for any prompt-based reasoning pipeline.

Key points

  • Logic benchmark: proofs need 4–12 rules, with 0, 5 or 10 distracting rules added. There are 200 problems per rule count and 15 order or distractor variants each, giving about 27K items. A proof counts as correct only if every step is valid.
  • Order variants: premise orders are labelled by normalized Kendall tau, from forward (τ = 1) through 0.5, 0 and −0.5 to backward (τ = −1).
  • Model differences:
    • The GPT models handle the reversed (backward-chaining) order better than random shuffles.
    • PaLM 2-L tends to perform worst on the reversed order.
  • R-GSM: 220 pairs of original and reordered GSM8K test problems, each with 5–8 sentences. All models lose accuracy on the reordered versions, especially on longer problems.
  • Ruling out context length: inputs are under 300 tokens, so the lost-in-the-middle effect does not explain the results. The authors leave fixes for the order effect to future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States567 works · 335,189 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,948 works

Cite

Cite as Xinyun Chen, Ryan A. Chi, Xuezhi Wang and Denny Zhou. 2024. Premise Order Matters in Reasoning with Large Language Models. arXiv:2402.08939.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2402.08939 · arXiv 2402.08939

@misc{chen2024premise,
  title = {{Premise Order Matters in Reasoning with Large Language Models}},
  author = {Xinyun Chen, Ryan A. Chi, Xuezhi Wang and Denny Zhou},
  year = {2024},
  doi = {10.48550/arxiv.2402.08939},
  eprint = {2402.08939},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.08939}
}

References

38 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Abdou, V. Ravishankar, A. Kulmizev, and A. Søgaard. Word order does matter and shuffled language models know it. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6907–6919, 2022. find
  2. J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021.
  3. L. Berglund, M. Tong, M. Kaufmann, M. Balesni, A. C. Stickland, T. Korbak, and O. Evans. The reversal curse: Llms trained on" a is b" fail to learn" b is a". arXiv preprint arXiv:2309.12288, 2023. find
  4. S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
  5. Q. Cao, T. Kojima, Y. Matsuo, and Y. Iwasawa. Unnatural error correction: Gpt-4 can almost perfectly handle unnatural scrambled text. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8898–8913, 2023. find
  6. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
  7. V. A. Cicirello. Kendall tau sequence distance: Extending kendall tau from ranks to sequences. arXiv preprint arXiv:1905.02752, 2019. — [arXiv:1905.02752](https://arxiv.org/abs/1905.02752) find
  8. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
  9. M. Dekeyser, W. Schroyens, W. Schaeken, O. Spitaels, and G. d’Ydewalle. Preferred premise order in propositional reasoning: Semantic informativeness and co-reference. Deductive reasoning and strategies, pages 73–95, 2000. find
  10. D. Ferreira and A. Freitas. Premise selection in natural language mathematical texts. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7365–7374, 2020.
  11. Gemini. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023.
  12. V. Girotto, A. Mazzocco, and A. Tasso. The effect of premise order in conditional reasoning: A test of the mental model theory. Cognition, 63(1):1–28, 1997. find
  13. Google. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023.
  14. T. Hagendorff, S. Fabi, and M. Kosinski. Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in chatgpt. Nature Computational Science, 3(10):833–838, 2023. find
  15. S. Han, H. Schoelkopf, Y. Zhao, Z. Qi, M. Riddell, L. Benson, L. Sun, E. Zubova, Y. Qiao, M. Burtell, et al. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv:2209.00840, 2022.
  16. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021.
  17. G. Irving, C. Szegedy, A. A. Alemi, N. Eén, F. Chollet, and J. Urban. Deepmath-deep sequence models for premise selection. Advances in neural information processing systems, 29, 2016. — DeepMath - Deep Sequence Models for Premise Selection
  18. E. Jones and J. Steinhardt. Capturing failures of large language models via human cognitive biases. Advances in Neural Information Processing Systems, 35:11785–11799, 2022. find
  19. N. Lee, K. Sreenivasan, J. D. Lee, K. Lee, and D. Papailiopoulos. Teaching arithmetic to small transformers. arXiv preprint arXiv:2307.03381, 2023.
  20. Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, et al. Competition-level code generation with alphacode. Science, 378(6624):1092– 1097, 2022.
  21. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024.
  22. R. T. McCoy, S. Yao, D. Friedman, M. Hardy, and T. L. Griffiths. Embers of autoregression: Understanding large language models through the problem they are trained to solve. arXiv preprint arXiv:2309.13638, 2023. find
  23. OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
  24. A. Saparov and H. He. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. arXiv preprint arXiv:2210.01240, 2022.
  25. A. Saparov, R. Y. Pang, V. Padmakumar, N. Joshi, S. M. Kazemi, N. Kim, and H. He. Testing the general deductive reasoning capacity of large language models using ood examples. arXiv preprint arXiv:2305.15269, 2023.
  26. P. K. Sen. Estimates of the regression coefficient based on kendall’s tau. Journal of the American statistical association, 63(324):1379–1389, 1968. find
  27. F. Shi, X. Chen, K. Misra, N. Scales, D. Dohan, E. H. Chi, N. Schärli, and D. Zhou. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pages 31210–31227. PMLR, 2023.
  28. K. Sinha, P. Parthasarathi, J. Pineau, and A. Williams. Unnatural language inference. arXiv preprint arXiv:2101.00010, 2020.
  29. Y. Wan, W. Wang, Y. Yang, Y. Yuan, J.-t. Huang, P. He, W. Jiao, and M. R. Lyu. A & b== b & a: Triggering logical reasoning failures in large language models. arXiv preprint arXiv:2401.00757, 2024. — [arXiv:2401.00757](https://arxiv.org/abs/2401.00757) find
  30. M. Wang, Y. Tang, J. Wang, and J. Deng. Premise selection for theorem proving by deep graph embedding. Advances in neural information processing systems, 30, 2017.
  31. P. Wang, L. Li, L. Chen, Z. Cai, D. Zhu, B. Lin, Y. Cao, Q. Liu, T. Liu, and Z. Sui. Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926, 2023. find
  32. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022.
  33. F. Xu, Q. Lin, J. Han, T. Zhao, J. Liu, and E. Cambria. Are large language models really good logical reasoners? a comprehensive evaluation from deductive, inductive and abductive views. arXiv preprint arXiv:2306.09841, 2023.
  34. S. Yan, C. Shen, J. Liu, and J. Ye. Concise and organized perception facilitates large language models for deductive reasoning. arXiv preprint arXiv:2310.03309, 2023. — [arXiv:2310.03309](https://arxiv.org/abs/2310.03309) find
  35. H. Zhang, L. H. Li, T. Meng, K.-W. Chang, and G. V. d. Broeck. On the paradox of learning to reason from data. arXiv preprint arXiv:2205.11502, 2022. — [arXiv:2205.11502](https://arxiv.org/abs/2205.11502) find
  36. H. Zhou, A. Bradley, E. Littwin, N. Razin, O. Saremi, J. Susskind, S. Bengio, and P. Nakkiran. What algorithms can transformers learn? a study in length generalization. arXiv preprint arXiv:2310.16028, 2023.
  37. Y. Zhou, U. Alon, X. Chen, X. Wang, R. Agarwal, and D. Zhou. Transformers can achieve length generalization but not robustly. arXiv preprint arXiv:2402.09371, 2024.
  38. Z. Zhu, Y. Xue, X. Chen, D. Zhou, J. Tang, D. Schuurmans, and H. Dai. Large language models can learn rules. arXiv preprint arXiv:2310.07064, 2023. — [arXiv:2310.07064](https://arxiv.org/abs/2310.07064) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2024-premise-order-matters-reasoning-large/