Elamur Research⌕
☰
home / papers / 2026 / Logical Phase Transitions: Understanding Collapse in LLM Logical Reasoning

Logical Phase Transitions: Understanding Collapse in LLM Logical Reasoning

Paper · 2026 · Xinglang Zhang, Yunyao Zhang, ZeLiang Chen et al.

Summary

The paper asks how the symbolic (first-order logic) reasoning of LLMs changes as problems become logically harder. The authors define the Logical Complexity Metric (LoCM), a single score per instance. It combines weighted counts of logical operators, which depend on nesting depth and the number of premises, with the number of reasoning hops, passed through a monotone transformation. To compute LoCM they built NSA-LR, which re-annotates existing logic benchmarks with paired natural-language and full FOL versions of premises, questions and reasoning chains. Plotting accuracy against LoCM for many open and closed models gives a consistent pattern. Accuracy holds roughly steady over a range, then falls sharply in narrow intervals, which the authors call Logical Phase Transitions. Past the last interval, accuracy drops to chance. They then propose Neuro-Symbolic Curriculum Tuning, which first aligns NL and FOL representations and then trains through stages of increasing complexity. Across five benchmarks this modestly reduces collapse on hard instances. The work matters because it replaces rough hop-count difficulty with a finer diagnostic of where LLM logical reasoning breaks.

Key points

  • LoCM covers the operators ∧, ∨, ¬, ⊕, →, ↔, ∀ and ∃, plus hop count. A square-root transformation fit accuracy best, and the full metric correlates with accuracy at about −0.391.
  • To build NSA-LR, GPT-5 and Qwen3-Max translated each statement independently. Matching translations passed CFG validation, and disagreements were resolved by hand. It is compared against ProntoQA, ProofWriter, FOLIO and ProverQA.
  • Every evaluated model showed one or two collapse intervals, after which accuracy approached the 1/3 random baseline. Gemma 3 1B failed once LoCM exceeded about 8, while larger Gemma 3 models stayed more robust overall.
  • Neuro-Symbolic Curriculum Tuning (a mixed-semantics model θ_MIX followed by Easy/Medium/Hard curriculum stages) raised average accuracy by +1.26 with naive prompting and +3.95 with CoT. It also generalized better to unseen logical compositions.
  • Fine-tuning and CoT (tested on Qwen2.5-7B) raised accuracy within each regime but did not move the transition thresholds. These thresholds also have to be calibrated separately for each model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Huazhong University of Science and TechnologyWuhan · China45 works · 6,345 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksReasoning · 1,426 works

Cite

Cite as Xinglang Zhang, Yunyao Zhang, ZeLiang Chen, Junqing Yu, Wei Yang and Zikai Song. 2026. Logical Phase Transitions: Understanding Collapse in LLM Logical Reasoning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.858

@inproceedings{zhang2026logical,
  title = {{Logical Phase Transitions: Understanding Collapse in LLM Logical Reasoning}},
  author = {Xinglang Zhang, Yunyao Zhang, ZeLiang Chen, Junqing Yu, Wei Yang and Zikai Song},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.858},
  url = {https://doi.org/10.18653/v1/2026.acl-long.858}
}

References

66 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Shengnan An, Xunliang Cai, Xuezhi Cao, Xiaoyu Li, Yehao Lin, Junlin Liu, Xinxuan Lv, Dan Ma, Xuanlin Wang, Ziwen Wang, and 1 others. 2025. Amo-bench: Large language models still struggle in high school math competitions. arXiv preprint arXiv:2510.26768. find
  3. Steven Bird. 2006. Nltk: the natural language toolkit. In Proceedings of the COLING/ACL 2006 interactive presentation sessions, pages 69–72.
  4. Xi Chen and Min Zeng. 2025. Prototype conditioned generative replay for continual learning in NLP. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Albuquerque, New Mexi… find
  5. Yao Chen, Yilong Chen, Yinqi Yang, Junyuan Shang, Zhenyu Zhang, Zefeng Zhang, Shuaiyi Nie, Shuohuan Wang, Yu Sun, Hua Wu, HaiFeng Wang, and Tingwen Liu. 2026. Sparse growing transformer: Training-time sparse depth allocation via progressive attention looping. Preprint, arXiv:2603.23998. find
  6. Yao Chen, Jiawei Sheng, Wenyuan Zhang, and Tingwen Liu. 2025. Improving reasoning capabilities in small models through mixture-of-layers distillation with stepwise attention on key information. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 4952–4971…
  7. Michael Cohen, Badri Vellambi, and Marcus Hutter. 2020. Asymptotically unambitious artificial general intelligence. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 2467– 2476. find
  8. Ben Eisner, Yi Yang, Todor Davchev, Mel Vecerik, Jonathan Scholz, and David Held. 2024. Deep se (3)-equivariant geometric reasoning for precise placement tasks. arXiv preprint arXiv:2404.13478. — [arXiv:2404.13478](https://arxiv.org/abs/2404.13478) find
  9. Zhen Fang, Zhuoyang Liu, Jiaming Liu, Hao Chen, Yu Zeng, Shiting Huang, Zehui Chen, Lin Chen, Shanghang Zhang, and Feng Zhao. 2025. Dualvla: Building a generalizable embodied agent via partial decoupling of reasoning and action. arXiv preprint arXiv:2511.22134. — [arXiv:2511.22134](https://arxiv.org/abs/2511.22134) find
  10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
  11. Ruiyan Han, Zhen Fang, XinYu Sun, Yuchen Ma, Ziheng Wang, Yu Zeng, Zehui Chen, Lin Chen, Wenxuan Huang, Wei-Jie Xu, and 1 others. 2026. Unicorn: Towards self-improving unified multimodal models through self-generated supervision. arXiv preprint arXiv:2601.03193.
  12. Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, and 1 others. 2022. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv:2209.00840.
  13. Zhezheng Hao, Hong Wang, Haoyang Liu, Jian Luo, Jiarui Yu, Hande Dong, Qiang Lin, Can Wang, and Jiawei Chen. 2025. Rethinking entropy interventions in rlvr: An entropy change perspective. arXiv preprint arXiv:2510.10150.
  14. Zhezheng Hao, Hong Wang, Jian Luo, Jianqing Zhang, Yuyan Zhou, Qiang Lin, Can Wang, Hande Dong, and Jiawei Chen. 2026. Recreate: Reasoning and creating domain agents driven by experience. arXiv preprint arXiv:2601.11100.
  15. Yangliu Hu, Zikai Song, Na Feng, Yawei Luo, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. 2025. Sf2t: Self-supervised fragment finetuning of video-llms for fine-grained understanding. arXiv preprint arXiv:2504.07745. — [arXiv:2504.07745](https://arxiv.org/abs/2504.07745) find
  16. Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403.
  17. Lev Davidovich Landau and 1 others. 1937. On the theory of phase transitions. Zh. eksp. teor. Fiz, 7(19-32):926. find
  18. Wenbing Li, Zikai Song, Hang Zhou, Yunyao Zhang, Junqing Yu, and Wei Yang. 2025a. Lora-mixer: Coordinate modular lora experts through serial attention routing. arXiv preprint arXiv:2507.00029. — [arXiv:2507.00029](https://arxiv.org/abs/2507.00029) find
  19. Wenbing Li, Hang Zhou, Junqing Yu, Zikai Song, and Wei Yang. 2024. Coupled mamba: Enhanced multimodal fusion with coupled state space model. arXiv preprint arXiv:2405.18014. find
  20. Yu Li, Tian Lan, and Zhengling Qi. 2025b. Inspo: Unlocking intrinsic self-reflection for llm preference optimization. arXiv preprint arXiv:2512.23126. — [arXiv:2512.23126](https://arxiv.org/abs/2512.23126) find
  21. Yu Li, Tian Lan, and Zhengling Qi. 2026a. When right meets wrong: Bilateral context conditioning with reward-confidence correction for grpo. arXiv preprint arXiv:2603.13134. — [arXiv:2603.13134](https://arxiv.org/abs/2603.13134) find
  22. Yu Li, Rui Miao, Zhengling Qi, and Tian Lan. 2026b. Arise: Agent reasoning with intrinsic skill evolution in hierarchical reinforcement learning. arXiv preprint arXiv:2603.16060. — [arXiv:2603.16060](https://arxiv.org/abs/2603.16060) find
  23. Honglin Lin, Zheng Liu, Yun Zhu, Chonghan Qin, Juekai Lin, Xiaoran Shang, Conghui He, Wentao Zhang, and Lijun Wu. 2026a. Mmfinereason: Closing the multimodal reasoning gap via open data-centric methods. arXiv preprint arXiv:2601.21821. — [arXiv:2601.21821](https://arxiv.org/abs/2601.21821) find
  24. Honglin Lin, Chonghan Qin, Zheng Liu, Qizhi Pei, Yu Li, Zhanping Zhong, Xin Gao, Yanfeng Wang, Conghui He, and Lijun Wu. 2026b. Scientific image synthesis: Benchmarking, methodologies, and downstream utility. arXiv preprint arXiv:2601.17027. — [arXiv:2601.17027](https://arxiv.org/abs/2601.17027) find
  25. Peter Smith. 2003. An introduction to formal logic. Cambridge University Press. find
  26. Guoming Ling, Zhongzhan Huang, Yupei Lin, Junxin Li, Shanshan Zhong, Hefeng Wu, and Liang Lin. 2026. Neural chain-of-thought search: Searching the optimal reasoning path to enhance large language models. Preprint, arXiv:2601.11340. find
  27. Zikai Song, Ying Tang, Run Luo, Lintao Ma, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. 2024a. Au-togenic language embedding for coherent point tracking. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 2021–2030. find
  28. Zheng Liu, Honglin Lin, Chonghan Qin, Xiaoyang Wang, Xin Gao, Yu Li, Mengzhang Cai, Yun Zhu, Zhanping Zhong, Qizhi Pei, and 1 others. 2026. Chartverse: Scaling chart reasoning via reliable programmatic synthesis from scratch. arXiv preprint arXiv:2601.13606.
  29. Zikai Song, Ying Tang, Run Luo, Lintao Ma, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. 2024b. Au-togenic language embedding for coherent point tracking. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 2021–2030. find
  30. Zheng Liu, Mengjie Liu, Jingzhou Chen, Jingwei Xu, Bin Cui, Conghui He, and Wentao Zhang. 2025. Fusion: Fully integration of vision-language representations for deep cross-modal understanding. arXiv preprint arXiv:2504.09925. — [arXiv:2504.09925](https://arxiv.org/abs/2504.09925) find
  31. Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. 2025. CoT-valve: Length-compressible chain-of-thought tuning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6025–6035, Vienna, Austria. Association for Co…
  32. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. arXiv preprint arXiv:2305.12295. the strengths and limitations of reasoning models via the lens of problem complexity. In NeurIPS.
  33. Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang, and Xinchao Wang. 2026. Hypergraph-state collaborative reasoning for multi-object tracking. Preprint, arXiv:2604.12665. — [arXiv:2604.12665](https://arxiv.org/abs/2604.12665) find
  34. Hongda Sun, Weikai Xu, Wei Liu, Jian Luan, Bin Wang, Shuo Shang, Ji-Rong Wen, and Rui Yan. 2024. Determlr: Augmenting llm-based logical reasoning from indeterminacy to determinacy. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag… find
  35. Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. 2020. Proofwriter: Generating implications, proofs, and abductive statements over natural language. arXiv preprint arXiv:2012.13048.
  36. Richard W Paul. 2013. Critical and reflective thinking: A philosophical perspective. In Dimensions of thinking and cognitive instruction, pages 445–494. Routledge. find
  37. Xingwei Tan, Marco Valentino, Mahmud Elahi Akhter, Maria Liakata, and Nikolaos Aletras. 2025. Enhancing logical reasoning in language models via symbolically-guided Monte Carlo process supervision. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31874… find
  38. Chengwen Qi, Ren Ma, Bowen Li, He Du, Binyuan Hui, Jinwang Wu, Yuanjun Laili, and Conghui He. 2025. Large language models meet symbolic provers for logical reasoning evaluation. arXiv preprint arXiv:2502.06563. — [arXiv:2502.06563](https://arxiv.org/abs/2502.06563) find
  39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  40. Bertrand Russell. 2020. Principles of mathematics. Routledge. find
  41. Kaiyang Wan, Lang Gao, Honglin Mu, Preslav Nakov, Yuxia Wang, and Xiuying Chen. 2025. A fano-style accuracy upper bound for llm single-pass reasoning in multi-hop qa. arXiv preprint arXiv:2509.21199. find
  42. Hyun Ryu, Gyeongman Kim, Hyemin S Lee, and Eunho Yang. 2024. Divide and translate: Compositional first-order logic translation and verification for complex logical reasoning. arXiv preprint arXiv:2410.08047. — [arXiv:2410.08047](https://arxiv.org/abs/2410.08047) find
  43. Abulhair Saparov and He He. 2022. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. arXiv preprint arXiv:2210.01240.
  44. Parshin Shojaee*, Iman Mirzadeh*, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. 2025. The illusion of thinking: Understanding Chengbing Wang, Yang Zhang, Wenjie Wang, Xiaoyan Zhao, Fuli Feng, Xiangnan He, and Tat-Seng Chua. 2026a. Think-while-generating: On-the-fly reasoning… — [arXiv:2512.06690](https://arxiv.org/abs/2512.06690) find
  45. Chengbing Wang, Wuqiang Zheng, Yang Zhang, Fengbin Zhu, Junyi Cheng, Yi Xie, Wenjie Wang, and Fuli Feng. 2026b. Perm: Psychology-grounded empathetic reward modeling for large language models. Preprint, arXiv:2601.10532.
  46. Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. 2023. Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning. arXiv preprint arXiv:2310.03731.
  47. Weiqi Wang, Tianqing Fang, Chunyang Li, Haochen Shi, Wenxuan Ding, Baixuan Xu, Zhaowei Wang, Jiaxin Bai, Xin Liu, Cheng Jiayang, Chunkit Chan, and Yangqiu Song. 2024. CANDLE: Iterative conceptualization and instantiation distillation from large language models for commonsense reasoning. In Proceedin… find
  48. Xin Wang, Yudong Chen, and Wenwu Zhu. 2021. A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence, 44(9):4555–4576. find
  49. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837. with large language models. Advances in neura…
  50. Liliang Ye, Yunyao Zhang, Yafeng Wu, Yi-Ping Phoebe Chen, Junqing Yu, Wei Yang, and Zikai Song. 2025. Mvp: Winning solution to smp challenge 2025 video track. arXiv preprint arXiv:2507.00950. — [arXiv:2507.00950](https://arxiv.org/abs/2507.00950) find
  51. Yu Zeng, Wenxuan Huang, Zhen Fang, Shuang Chen, Yufan Shen, Yishuo Cai, Xiaoman Wang, Zhenfei Yin, Lin Chen, Zehui Chen, and 1 others. 2026. Vision-deepresearch benchmark: Rethinking visual and textual search for multimodal large language models. arXiv preprint arXiv:2602.02185. — [arXiv:2602.02185](https://arxiv.org/abs/2602.02185) find
  52. Peixuan Zhang, Zijian Jia, Kaiqi Liu, Shuchen Weng, Si Li, and Boxin Shi. 2025a. Stage: Storyboard-anchored generation for cinematic multi-shot narrative. arXiv preprint arXiv:2512.12372. — [arXiv:2512.12372](https://arxiv.org/abs/2512.12372) find
  53. Peixuan Zhang, Chang Zhou, Ziyuan Zhang, Hualuo Liu, Chunjie Zhang, Jingqi Liu, Xiaohui Zhou, Xi Chen, Shuchen Weng, Si Li, and Boxin Shi. 2026. A benchmark and multi-agent system for instruction-driven cinematic video compilation. arXiv preprint arXiv:2604.10456. — [arXiv:2604.10456](https://arxiv.org/abs/2604.10456) find
  54. Tractatus logico-Yifan Zhang, Jingqin Yang, Yang Yuan, and Andrew Chi-Chih Yao. 2023. Cumulative reasoning with large language models. arXiv preprint arXiv:2308.04371. find
  55. Jundong Xu, Hao Fei, Meng Luo, Qian Liu, Liangming Pan, William Yang Wang, Preslav Nakov, Mong-Li Lee, and Wynne Hsu. 2024a. Aristotle: Mastering logical reasoning with a logic-complete decompose-search-resolve framework. arXiv preprint arXiv:2412.16953.
  56. Yunyao Zhang, Zikai Song, Hang Zhou, Wenfeng Ren, Yi-Ping Phoebe Chen, Junqing Yu, and Wei Yang. 2025b. ga − s 3 : Comprehensive social network simulation with group agents. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8950–8970, Vienna, Austria. Association for Comp… find
  57. Ludwig Wittgenstein. 2023. philosophicus. find
  58. Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. 2024b. Faithful logical reasoning via symbolic chain-of-thought. arXiv preprint arXiv:2405.18357.
  59. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025a. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  60. An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, and 1 others. 2025b. Qwen2. 5-1m technical report. arXiv preprint arXiv:2501.15383. — Qwen2.5-1M Technical Report
  61. Yuan Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, and Faramarz Fekri. 2023. Harnessing the power of large language models for natural language to first-order logic translation. arXiv preprint arXiv:2305.15541. find
  62. Yunyao Zhang, Xinglang Zhang, Junxi Sheng, Wenbing Li, Junqing Yu, Wei Yang, and Zikai Song. 2025c. From ambiguity to verdict: A semiotic-grounded multi-perspective agent for llm logical reasoning. Preprint, arXiv:2509.24765. — [arXiv:2509.24765](https://arxiv.org/abs/2509.24765) find
  63. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223, 1(2).
  64. Zi’ou Zheng, Christopher Malon, Martin Renqiang Min, and Xiaodan Zhu. 2024. Exploring the role of reasoning structures for constructing proofs in multi-step natural language reasoning with large language models. arXiv preprint arXiv:2410.08436. find
  65. Yixiao Zhou, Yang Li, Dongzhou Cheng, Hehe Fan, and Yu Cheng. 2026. Look inward to explore outward: Learning temperature policy from llm internal states via hierarchical rl. arXiv preprint arXiv:2602.13035. — [arXiv:2602.13035](https://arxiv.org/abs/2602.13035) find
  66. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-logical-phase-transitions-understanding-collapse/