Elamur Research⌕
☰
home / papers / 2023 / Solving Math Word Problems via Cooperative Reasoning induced Language Models

Solving Math Word Problems via Cooperative Reasoning induced Language Models

Paper · 2023 · Xinyu Zhu, Junjie Wang, Lin Zhang et al.

Summary

Pre-trained language models often fail on math word problems because they generate text without step-by-step supervision. Prompting methods such as Chain-of-Thought mostly help only very large models. Inspired by dual-process theory from cognitive science, the authors propose Cooperative Reasoning (CoRe). A generator acts as the fast, intuitive System 1, and verifiers act as the deliberate System 2. The pipeline has three stages. First, the generator is fine-tuned on GSM8K questions with annotated rationales. It then samples many candidate solutions, which are used to train two verifiers. Second, at inference time, Monte Carlo Tree Search lets the verifier scores steer decoding while a solution is still being built, rather than only ranking finished answers. Third, a self-thinking loop keeps high-scoring generated solutions and adds them back to the training data. CoRe was tested in zero-shot and fine-tuning settings on several arithmetic benchmarks. It outperforms baselines dozens of times larger, gaining up to 9.6% on MultiArith. This shows that feedback-guided search can partly replace sheer model scale.

Key points

  • Generator and step verifier. The generator (System 1) is trained with a standard language-modeling loss. The step verifier V_step is a token-level scorer trained jointly on language modeling and per-token MSE regression toward answer correctness.
  • Path verifier. V_path is a BERT-like model that scores a whole reasoning path from its [CLS] representation. The overall verifier objective combines the step loss, the path loss and the language-modeling loss.
  • Guided search. Cooperative inference uses MCTS with a modified PUCT selection rule. This gives feedback on partial reasoning paths during generation, not just on final answers.
  • Training data. Fine-tuning and self-thinking rely only on GSM8K, which has about 7.5K training examples. Zero-shot results on other arithmetic datasets suggest that reasoning skill transfers within the domain.
  • Limitations and code. The method works with models of only a few billion parameters, but it was not tested on GPT-3 or PaLM. Code is available at github.com/TianHongZXY/CoRe.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tsinghua UniversityBeijing · China137 works · 83,262 citations🎓Chinese University of Hong KongHong Kong · China53 works · 12,921 citations🎓Waseda UniversityTokyo · Japan4 works · 531 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Mathematics · 140 works Applied Mathematics · 21 works

🧭 Topics 2 Language models & pretraining · 1,343 works Reasoning · 510 works

Cite

Cite as Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang and Yujiu Yang. 2023. Solving Math Word Problems via Cooperative Reasoning induced Language Models. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-long.245 · arXiv 2210.16257

@inproceedings{zhu2023solving,
  title = {{Solving Math Word Problems via Cooperative Reasoning induced Language Models}},
  author = {Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang and Yujiu Yang},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.245},
  eprint = {2210.16257},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.acl-long.245}
}

References

30 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akhter Al Amin and Kazi Sinthia Kabir. 2022. A disability lens towards biases in GPT-3 generated open-ended languages. CoRR, abs/2206.11993. — [arXiv:2206.11993](https://arxiv.org/abs/2206.11993) find
  2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… our card
  3. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… our card
  4. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. CoRR, abs/2110.14168. our card
  5. Johannes Czech, Patrick Korus, and Kristian Kersting. 2021. Improving alphazero using monte-carlo graph search. In ICAPS, pages 103–111. AAAI Press. find
  6. Jonathan St.B.T. Evans. 2003. In two minds: dual-process accounts of reasoning. Trends in Cognitive Sciences, 7(10):454–459. find
  7. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: decoding-enhanced bert with disentangled attention. In ICLR. OpenReview.net. our card
  8. Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux. find
  9. Levente Kocsis and Csaba Szepesvári. 2006. Bandit based monte-carlo planning. In ECML, volume 4212 of Lecture Notes in Computer Science, pages 282–293. Springer. find
  10. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. CoRR, abs/2205.11916. our card
  11. Rik Koncel-Kedziorski, Hannaneh Hajishirzi, Ashish Sabharwal, Oren Etzioni, and Siena Dumas Ang. 2015. Parsing algebraic word problems into equations. Transactions of the Association for Computational Linguistics, 3:585–597. find
  12. Sanjay Krishnan, Jiannan Wang, Michael J. Franklin, Ken Goldberg, and Tim Kraska. 2016. Privateclean: Data cleaning and differential privacy. In SIGMOD Conference, pages 937–951. ACM. find
  13. Yihuai Lan, Lei Wang, Qiyuan Zhang, Yunshi Lan, Bing Tian Dai, Yan Wang, Dongxiang Zhang, and Ee-Peng Lim. 2022. Mwptoolkit: An open-source framework for deep learning-based math word problem solvers. In AAAI, pages 13188–13190. AAAI Press. find
  14. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2022. On the advance of making language models better reasoners. CoRR, abs/2206.02336. — [arXiv:2206.02336](https://arxiv.org/abs/2206.02336) find
  15. Chao-Chun Liang, Kuang-Yi Hsu, Chien-Tsung Huang, Chung-Min Li, Shen-Yu Miao, and Keh-Yih Su. 2016. A tag-based statistical english math word problem solver with understanding, reasoning and explanation. In IJCAI, pages 4254–4255. IJCAI/AAAI Press. find
  16. Li Lucy and David Bamman. 2021. Gender and representation bias in GPT-3 generated stories. In Proceedings of the Third Workshop on Narrative Understanding, pages 48–55, Virtual. Association for Computational Linguistics. our card
  17. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020. A diverse corpus for evaluating and developing English math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 975–984, Online. Association for Computational Linguistics. find
  18. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2022. Show your work: Scratchpads for intermediate computation with language models. In Deep Learning for Co… our card
  19. Maxwell I. Nye, Michael Henry Tessler, Joshua B. Tenenbaum, and Brenden M. Lake. 2021. Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. In NeurIPS, pages 25192–25204. find
  20. Subhro Roy and Dan Roth. 2015. Solving general arithmetic word problems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pages 1743–1752. The Association for Computational Linguistics. our card
  21. Subhro Roy and Dan Roth. 2017. Unit dependency graph and its application to arithmetic word problem solving. In AAAI, pages 3082–3088. AAAI Press. our card
  22. Subhro Roy, Tim Vieira, and Dan Roth. 2015. Reasoning about quantities in natural language. Trans. Assoc. Comput. Linguistics, 3:1–13. find
  23. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan C… our card
  24. Thomas Scialom, Paul-Alexis Dray, Jacopo Staiano, Sylvain Lamprier, and Benjamin Piwowarski. 2021. To beam or not to beam: That is a question of cooperation for language gans. In NeurIPS, pages 26585– 26597. find
  25. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzho… our card
  26. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/ mesh-transformer-jax. ↗
  27. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. CoRR, abs/2203.11171. our card
  28. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022a. Finetuned language models are zero-shot learners. In ICLR. OpenReview.net. find
  29. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022b. Emergent abilities of large language models. CoRR, abs/2206.07… our card
  30. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. 2022c. Chain of thought prompting elicits reasoning in large language models. CoRR, abs/2201.11903. our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhu-2023-solving-math-word-problems-cooperative/