Elamur Research⌕
☰
home / papers / 2025 / Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions

Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions

Paper · 2025 · Leonardo Ranaldi, Marco Valentino and Andre Freitas

Summary

Chain-of-Thought (CoT) prompting helps LLMs reason step by step, but its explanations are prone to content biases that hurt robustness and faithfulness. Fully symbolic pipelines with external solvers address this, but they require a complete translation of the problem into a formal language, which reduces efficiency and flexibility. The authors propose a middle path called QuaSAR (Quasi-Symbolic Abstract Reasoning). It is a CoT variant in which the model formalises only the variables and predicates that matter and keeps the rest in natural language. This separates logical structure from world content. The method runs as a single prompt with four stages: abstraction, formalisation, explanation and answering. It is used in two ways: as an in-context strategy for strong models such as GPT-4o, and as a generator of synthetic demonstrations for fine-tuning smaller Llama3 and Qwen-2 models. Evaluation covers maths, reasoning and NLU benchmarks, including adversarial variants such as MMLU-Redux and GSM-Symbolic. QuaSAR raises accuracy by up to roughly 8% over standard CoT and yields more consistent reasoning. This matters because it offers a cheap, solver-free way to make LLM reasoning more robust.

Key points

  • QuaSAR has four stages:
    • Abstraction: identify predicates, variables and constants.
    • Formalisation: restate the problem in mixed symbolic and natural-language form.
    • Explanation: build a quasi-symbolic reasoning chain.
    • Answering: produce the final result in a fixed answer template.
  • On GPT-4o, average accuracy rises by 8% over CoT, 6.8% over CoMAT and 8.2% over Faithful CoT.
  • When used for demonstration tuning, Llama-3-8B beats its larger counterpart with only 50% of the training data, and Qwen2-7B does so with only 25%. QuaSAR also outperforms other supervised tuning approaches.
  • QuaSAR needs no external symbolic solver and only one prompting step. Its structured rationales also turned out to be easier for models to self-correct.
  • The authors plan to apply the method to non-English data via a multilingual extension of GSM-Symbolic.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom359 works · 77,352 citations🎓University of SheffieldSheffield · United Kingdom113 works · 21,542 citations🎓University of ManchesterManchester · United Kingdom87 works · 44,938 citations🔬Idiap Research InstituteMartigny-Combe · Switzerland32 works · 1,954 citations🤝Cancer Research UKLondon · United Kingdom2 works · 42 citations

Also on the byline: Cancer Research UK Manchester Institute.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Leonardo Ranaldi, Marco Valentino and Andre Freitas. 2025. Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.acl-long.843

@inproceedings{ranaldi2025improving,
  title = {{Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions}},
  author = {Leonardo Ranaldi, Marco Valentino and Andre Freitas},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.843},
  url = {https://doi.org/10.18653/v1/2025.acl-long.843}
}

References

60 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Aaron Grattafiori et al. 2024. The llama 3 herd of models.
  3. Alfonso Amayuelas, Shuai Zhang, Xi Susie Rao, and Ce Zhang. 2022. Neural methods for logical reasoning over knowledge graphs. In International Conference on Learning Representations. find
  4. Erik Arakelyan, Pasquale Minervini, Pat Verga, Patrick Lewis, and Isabelle Augenstein. 2024. Flare: Faithful logic-aided reasoning and exploration. find
  5. Hugo Bronkhorst, Gerrit Roorda, Cor Suhre, and Martin Goedhart. 2019. Logical reasoning in formal and everyday reasoning tasks. International Journal of Science and Mathematics Education, 18(8):1673–1694. find
  6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  7. Denise D Cummins, Todd Lubart, Olaf Alksnis, and Robert Rist. 1991. Conditional reasoning and causation. Memory & cognition, 19:274–282. find
  8. Dhairya Dalal, Marco Valentino, Andre Freitas, and Paul Buitelaar. 2024. Inference to the best explanation in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 217–235, Bangkok, Thailand. Association for C… find
  9. Tuan Dinh, Jinman Zhao, Samson Tan, Renato Negrinho, Leonard Lausen, Sheng Zha, and George Karypis. 2023. Large language models of code fail at completing code with potential bugs. find
  10. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024. A survey on in-context learning.
  11. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistic…
  12. Deborah Ferreira and André Freitas. 2020. Premise selection in natural language mathematical texts. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7365– 7374.
  13. Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, et al. 2024. Are we done with MMLU? arXiv preprint arXiv:2406.04127. find
  14. Olga Gerasimova, Nikita Severin, and Ilya Makarov. 2023. Comparative analysis of logic reasoning and graph neural networks for ontology-mediated query answering with a covering axiom. IEEE Access. find
  15. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402.14008.
  16. Dongwei Jiang, Marcio Fonseca, and Shay B Cohen. 2024a. Leanreasoner: Boosting complex logical reasoning with lean. arXiv preprint arXiv:2403.13312. find
  17. Dongwei Jiang, Guoxuan Wang, Yining Lu, Andrew Wang, Jingyu Zhang, Chuyu Liu, Benjamin Van Durme, and Daniel Khashabi. 2024b. Rationalyst: Pre-training process-supervision for improving reasoning. find
  18. Zhuoxuan Jiang, Haoyuan Peng, Shanshan Feng, Fan Li, and Dongsheng Li. 2024c. LLMs can find mathematical reasoning mistakes by pedagogical chain-of-thought. arXiv preprint arXiv:2405.06705. — [arXiv:2405.06705](https://arxiv.org/abs/2405.06705) find
  19. Philip Kitcher. 1981. Explanatory unification. Philosophy of science, 48(4):507–531. find
  20. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213.
  21. Joshua Ong Jun Leang, Aryo Pradipta Gema, and Shay B. Cohen. 2024. Comat: Chain of mathematically annotated thought improves mathematical reasoning.
  22. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017. Program induction by rationale generation: Learning to solve and explain algebraic word problems. arXiv preprint arXiv:1705.04146. find
  23. Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, et al. 2024. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406.06592.
  24. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. Faithful chain-of-thought reasoning. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter…
  25. Jordan Meadows and André Freitas. 2023. Introduction to Mathematical Language Processing: Informal Proofs, Word Problems, and Supporting Tasks. Transactions of the Association for Computational Linguistics, 11:1162–1184. find
  26. Jordan Meadows, Marco Valentino, Damien Teney, and Andre Freitas. 2023. A symbolic framework for systematic evaluation of mathematical reasoning with transformers. find
  27. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267:1–38.
  28. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. 2024. Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models.
  29. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. arXiv preprint arXiv:2305.12295.
  30. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080–2094, Online. Associat…
  31. Nancy Pennington and Reid Hastie. 1993. Reasoning in explanation-based decision making. Cognition, 49(1):123–163. find
  32. Xin Quan, Marco Valentino, Louise Dennis, and Andre Freitas. 2024a. Enhancing ethical explanations of large language models through iterative symbolic refinement. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p… find
  33. Xin Quan, Marco Valentino, Louise A. Dennis, and André Freitas. 2024b. Verification and refinement of natural language explanations through llm-symbolic theorem proving.
  34. Leonardo Ranaldi and Andre Freitas. 2024a. Aligning large and small language models via chain-of-thought reasoning. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1812–1827.
  35. Leonardo Ranaldi and Andre Freitas. 2024b. Self-refine instruction-tuning for aligning reasoning in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2325–2347, Miami, Florida, USA. Association for Computational Linguistics.
  36. Leonardo Ranaldi and Giulia Pucci. 2025. Multilingual reasoning via self-training. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11566–11582, Albuquerque, New M… find
  37. Leonardo Ranaldi, Giulia Pucci, Barry Haddow, and Alexandra Birch. 2024a. Empowering multi-step reasoning across languages via program-aided language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12171–12187, Miami, Florida, USA. Association… find
  38. Leonardo Ranaldi, Giulia Pucci, Federico Ranaldi, Elena Sofia Ruzzetti, and Fabio Massimo Zanzotto. 2024b. A tree-of-thoughts to broaden multi-step reasoning across languages. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1229–1241, Mexico City, Mexico. Association… find
  39. Leonardo Ranaldi, Federico Ranaldi, Fabio Massimo Zanzotto, Barry Haddow, and Alexandra Birch. 2025a. Improving multilingual retrieval-augmented language models through dialectic reasoning argumentations.
  40. Leonardo Ranaldi, Marco Valentino, and Andre Freitas. 2025b. Eliciting critical reasoning in retrieval-augmented generation via contrastive explanations. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Tech… find
  41. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. 2023. Gpqa: A graduate-level google-proof q&a benchmark.
  42. John Alan Robinson. 1965. A machine-oriented logic based on the resolution principle. Journal of the ACM (JACM), 12(1):23–41. find
  43. Zhi Rui Tam, Cheng-Kuang Wu, Yi-Lin Tsai, Chieh-Yen Lin, Hung-yi Lee, and Yun-Nung Chen. 2024. Let me speak freely? a study on the impact of format restrictions on performance of large language models. arXiv preprint arXiv:2408.02442. — [arXiv:2408.02442](https://arxiv.org/abs/2408.02442) find
  44. Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2024. Language models don’t always say what they think: unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36.
  45. Marco Valentino and André Freitas. 2024. On the nature of explanation: An epistemological-linguistic perspective for explanation-based natural language inference. Philosophy & Technology, 37(3):88. find
  46. Marco Valentino, Mokanarangan Thayaparan, Deborah Ferreira, and André Freitas. 2022a. Hybrid autoregressive inference for scalable multi-hop explanation regeneration. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):11403–11411. find
  47. Marco Valentino, Mokanarangan Thayaparan, and André Freitas. 2021. Unification-based reconstruction of multi-hop explanations for science questions. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 200–211, Online. Ass… find
  48. Marco Valentino, Mokanarangan Thayaparan, and André Freitas. 2022b. Case-based abductive natural language inference. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1556–1568, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. find
  49. Hao Wang. 1954. The formalization of mathematics. Journal of Symbolic Logic, 19(4):241–266. find
  50. Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091. find
  51. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  52. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.
  53. Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. 2024. Faithful logical reasoning via symbolic chain-of-thought. arXiv preprint arXiv:2405.18357.
  54. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. find
  55. Xi Ye, Qiaochu Chen, Isil Dillig, and Greg Durrett. 2024. Satlm: Satisfiability-aided language models using declarative prompting. Advances in Neural Information Processing Systems, 36. find
  56. Evelyn Yee, Alice Li, Chenyu Tang, Yeon Ho Jung, Ramamohan Paturi, and Leon Bergen. 2024. Dissociation of faithful and unfaithful reasoning in LLMs. arXiv preprint arXiv:2405.15092. — [arXiv:2405.15092](https://arxiv.org/abs/2405.15092) find
  57. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493.
  58. Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen, Heng-Tze Cheng, Ed H. Chi, Quoc V Le, and Denny Zhou. 2024. Take a step back: Evoking reasoning via abstraction in large language models. In The Twelfth International Conference on Learning Representations. find
  59. Qihuang Zhong, Kang Wang, Ziyang Xu, Juhua Liu, Liang Ding, Bo Du, and Dacheng Tao. 2024. Achieving\> 97% on gsm8k: Deeply understanding the problems makes LLMs perfect reasoners. arXiv preprint arXiv:2404.14963. — [arXiv:2404.14963](https://arxiv.org/abs/2404.14963) find
  60. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023. Least-to-most prompting enables complex reasoning in large language models.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ranaldi-2025-improving-chain-thought-reasoning-quasi/