Elamur Research⌕
☰
home / papers / 2026 / Training Language Models to Use Prolog as a Tool

Training Language Models to Use Prolog as a Tool

Paper · 2026 · Niklas Mellgren, Peter Schneider-Kamp and Lukas Galke Poech

Summary

Language models often produce reasoning that sounds convincing but is wrong and hard to check. The authors investigate whether a small model can learn, through reinforcement learning, to offload reasoning to Prolog, whose logical derivations are explicit and inspectable. They fine-tune Qwen2.5-3B-Instruct with GRPO on a cleaned GSM8K variant, released as gsm8k-prolog-prover. Programs are executed in SWI-Prolog. The study varies three factors: four system prompts, three reward suites (combining execution, syntax, semantic and structural signals), and several inference protocols, from single-try to agentic self-repair.

GRPO beats a supervised fine-tuning baseline on GSM8K. With agentic inference, the 3B model's zero-shot scores on MMLU-STEM and MMLU-Pro are comparable to few-shot 7B models. The central finding is a tension between accuracy and auditability. When rewarded only for correct answers, the model reasons in natural language and uses Prolog merely to compute the final result, which the authors classify as reward hacking. Rewarding symbolic structure produces fully auditable programs but lowers accuracy. This matters for neurosymbolic systems in high-stakes fields such as medicine and law.

Key points

  • Setup: Qwen2.5-3B-Instruct, trained with GRPO, using SWI-Prolog as the execution engine. The cleaned training set is published as gsm8k-prolog-prover, and the code is in the aisilab/Prolog-as-a-Tool repository.
  • System prompts: SP-Base, SP-Struct, SP-Declare (every numeric constant must be a named predicate) and SP-Reflect (adds self-review before emitting code).
  • Reward suites:
    • Rwd1 covers correctness, syntax and format.
    • Rwd2 adds Sentence-BERT similarity and predicate-name overlap.
    • Rwd3 adds curriculum weighting and a penalty for hard-coded answers.
  • Inference protocols:
    • Multiple-try with external verification gives the best in-distribution accuracy.
    • Agentic-Internal, which keeps debugging context, generalizes best to MMLU and performs best on the full GSM8K test set.
  • Error analysis:
    • SP-Struct with Rwd1 reaches about 90% execution accuracy but produces structurally weak programs.
    • The Declare variants give 31–36% wrong answers.
    • Most failures are missing code extraction, followed by uninstantiated variables.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Niklas Mellgren, Peter Schneider-Kamp and Lukas Galke Poech. 2026. Training Language Models to Use Prolog as a Tool. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1644

@inproceedings{mellgren2026training,
  title = {{Training Language Models to Use Prolog as a Tool}},
  author = {Niklas Mellgren, Peter Schneider-Kamp and Lukas Galke Poech},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1644},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1644}
}

References

20 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In NeurIPS, pages 1171–1179. find
  2. Jan Betley, Niels Warncke, Anna Sztyber-Betley, Daniel Tan, Xuchan Bao, Martín Soto, Megha Srivastava, Nathan Labenz, and Owain Evans. 2026. Training large language models on narrow tasks can lead to broad misalignment. Nat., 649(8097):584–589. find
  3. Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul A. Szerlip, Paul Horsfall, and Noah D. Goodman. 2019. Pyro: Deep universal probabilistic programming. J. Mach. Learn. Res., 20:28:1–28:6. find
  4. Nasim Borazjanizadeh and Steven Piantadosi. 2024. Reliable reasoning beyond natural language. Preprint, arXiv:2407.11373. — [arXiv:2407.11373](https://arxiv.org/abs/2407.11373) find
  5. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. 2025. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. In ICML, Proceedings of Machine Learning Research. PMLR / OpenReview.net.
  6. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Deepseek-R1 incentivizes reasoning in LLMs through rein… find
  7. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In ICLR. OpenReview.net.
  8. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In ICLR. OpenReview.net.
  9. Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, Scott Emmons, Owain Evans, David Farhi, Ryan Greenblatt, Dan Hendrycks, Marius Hobbhahn, Evan Hubinger, Geoffrey Irving, Erik Jenner, and 22 others. 2025. Chain… find
  10. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, and 1 others. 2023. Measuring faithfulness in chain-of-thought reasoning. Preprint, arXiv:2307.13702.
  11. Monte MacDiarmid, Benjamin Wright, Jonathan Uesato, Joe Benton, Jonathan Kutasov, Sara Price, Naia Bouscal, Samuel R. Bowman, Trenton Bricken, Alex Cloud, Carson Denison, Johannes Gasteiger, Ryan Greenblatt, Jan Leike, Jack Lindsey, Vladimir Mikulik, Ethan Perez, Alex Rodrigues, Drake Thomas, and 3… — [arXiv:2511.18397](https://arxiv.org/abs/2511.18397) find
  12. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. 2025. s1: Simple test-time scaling. In EMNLP, pages 20275–20321.
  13. OpenAI. 2024. OpenAI o1 system card. Preprint, arXiv:2412.16720.
  14. Razvan Pascanu, Tomás Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In ICML, pages 1310–1318. JMLR.org. find
  15. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024a. Qwen2.5 technical report. Preprint, arXiv:24… find
  16. Debjit Paul, Robert West, Antoine Bosselut, and Boi Faltings. 2024. Making reasoning matter: Measuring and improving faithfulness of chain-of-thought reasoning. In EMNLP Findings, pages 15012–15032.
  17. Xiaocheng Yang, Bingsen Chen, and Yik-Cheung Tam. 2024b. Arithmetic reasoning with LLM: Prolog generation & permutation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 6… find
  18. Federico Pennino, Bianca Raimondi, Massimo Rondelli, Andrea Gurioli, and Maurizio Gabbrielli. 2025. From reasoning to code: GRPO optimization for underrepresented languages. Preprint, arXiv:2506.11027. — [arXiv:2506.11027](https://arxiv.org/abs/2506.11027) find
  19. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In ICLR. OpenReview.net.
  20. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. In NeurIPS.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mellgren-2026-training-language-models-use-prolog/