Elamur Research⌕
☰
home / papers / 2023 / Natural Language Reasoning, A Survey

Natural Language Reasoning, A Survey

Paper · 2023 · Fei Yu, Hongbo Zhang, Prayag Tiwari et al.

Summary

The survey addresses a terminology problem in NLP: the word "reasoning" is applied inconsistently. Some tasks labelled as reasoning, such as parts of commonsense reasoning, involve little actual inference. Others, like natural language inference or multi-hop QA, often require it but are not named that way. The authors draw on philosophy and logic to build an NLP-oriented definition of natural language reasoning. They also discuss which tasks genuinely demand it and propose a taxonomy of reasoning types. They then review more than two hundred works built on transformer-based PLMs. Methods are grouped into end-to-end, forward (e.g. chain-of-thought prompting) and backward reasoning. Benchmarks are surveyed for classical logical reasoning, NLI, multi-hop QA and commonsense reasoning. The scope is limited to unstructured, single-modality text and excludes neuro-symbolic methods, tables, knowledge triples and mathematical reasoning. The authors argue that backward reasoning deserves more attention because its smaller search space makes it more efficient. They highlight defeasible (non-deductive) reasoning as a key open direction. The result is a shared vocabulary and a structured map of the field.

Key points

  • The literature base comprises 200+ papers. They were located by keyword searches ("reasoning", "inference", "multi-hop", etc.) of the 2019–2022 proceedings of NeurIPS, ICML, ICLR, ACL, EMNLP and NAACL. A companion paper list is maintained at github.com/FreedomIntelligence/ReasoningNLP.
  • The proposed definition builds on two philosophical notions:
    • theoretical (assertion-based) reasoning, which derives conclusions from premises;
    • practical (action-based) reasoning, which infers actions from goals and knowledge.
  • Backward reasoning is said to help both medium-size PLMs and LLMs. In contrast, CoT prompting mainly benefits large models.
  • In the authors' case studies, ChatGPT still fails to be perfect even on single-step deduction. Yet it sometimes produced fuller answers than the gold annotations in EntailmentBank and WikiWhy.
  • The listed future directions are:
    • generalisation to longer reasoning chains;
    • defeasible reasoning;
    • reasoning in non-English languages, which is harder because of data sparsity;
    • multimodality;
    • faithfulness and interpretability;
    • planning;
    • self-correction;
    • LLM self-improvement.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Chinese University of Hong Kong, ShenzhenShenzhen · China56 works · 3,212 citations🎓Shenzhen Research Institute of Big DataShenzhen · China20 works · 2,016 citations🎓Halmstad UniversityHalmstad · Sweden4 works · 167 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Fei Yu, Hongbo Zhang, Prayag Tiwari and Benyou Wang. 2023. Natural Language Reasoning, A Survey. arXiv:2303.14725.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.14725 · arXiv 2303.14725

@misc{yu2023natural,
  title = {{Natural Language Reasoning, A Survey}},
  author = {Fei Yu, Hongbo Zhang, Prayag Tiwari and Benyou Wang},
  year = {2023},
  doi = {10.48550/arxiv.2303.14725},
  eprint = {2303.14725},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.14725}
}

References

192 references, 87 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. Aggarwal, D. Mandowara, V. Agrawal, D. Khandelwal, P. Singla, and D. Garg. Explanations for commonsenseqa: New dataset and models. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International…
  2. E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou. What learning algorithm is in-context learning? investigations with linear models. CoRR, abs/2211.15661, 2022.
  3. P. A. Angeles. Dictionary of Philosophy. Barnes & Noble Books, 1981. find
  4. Y. Bang, S. Cahyawijaya, N. Lee, W. Dai, D. Su, B. Wilie, H. Lovenia, Z. Ji, T. Yu, W. Chung, Q. V. Do, Y. Xu, and P. Fung. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. CoRR, abs/2302.04023, 2023.
  5. Q. Bao, A. Y. Peng, T. Hartill, N. Tan, Z. Deng, M. Witbrock, and J. Liu. Multi-step deductive reasoning over natural language: An empirical study on out-of-distribution generalisation. The 2nd International Joint Conference on Learning and Reasoning and 16th International Workshop on Neural-Symboli… find
  6. G. Betz, C. Voigt, and K. Richardson. Critical thinking for language models. In IWCS, pages 63–75. Association for Computational Linguistics, 2021. find
  7. C. Bhagavatula, R. L. Bras, C. Malaviya, K. Sakaguchi, A. Holtzman, H. Rashkin, D. Downey, W. Yih, and Y. Choi. Abductive commonsense reasoning. In ICLR. OpenReview.net, 2020.
  8. Y. Bisk, R. Zellers, R. L. Bras, J. Gao, and Y. Choi. PIQA: reasoning about physical commonsense in natural language. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI…
  9. S. Blackburn. The Oxford Dictionary of Philosophy. Oxford University Press, 2008. find
  10. M. Boratko, X. Li, T. O’Gorman, R. Das, D. Le, and A. McCallum. Protoqa: A question answering dataset for prototypical common-sense reasoning. In EMNLP (1), pages 1122–1136. Association for Computational Linguistics, 2020. find
  11. K. Bostrom, X. Zhao, S. Chaudhuri, and G. Durrett. Flexible generation of natural language deductions. In M. Moens, X. Huang, L. Specia, and S. W. Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican R… find
  12. S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning. A large annotated corpus for learning natural language inference. In L. Màrquez, C. Callison-Burch, J. Su, D. Pighin, and Y. Marton, editors, Proceedings of the 2015 Conference on Empirical Methods in Natural Language , Vol. 1, No. 1, Article . P…
  13. T. E. o. E. Britannica. inference. Encyclopedia Britannica, 16 Jun. 2017, 2017. https://www.britannica.com/topic/inference-reason. ↗
  14. T. E. o. E. Britannica. reason. Encyclopedia Britannica, 15 May. 2020, 2020. https://www.britannica.com/topic/reason. ↗
  15. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… — Language Models are Few-Shot Learners
  16. S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, H. Nori, H. Palangi, M. T. Ribeiro, and Y. Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4. CoRR, abs/2303.12712, 2023.
  17. K. Burton, A. Java, and I. Soboroff. The icwsm 2009 spinn3r dataset. In Third Annual Conference on Weblogs and Social Media (ICWSM 2009), 2009. find
  18. O. Camburu, T. Rocktäschel, T. Lukasiewicz, and P. Blunsom. e-snli: Natural language inference with natural language explanations. In S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference…
  19. T. Chakrabarty, D. Ghosh, A. Poliak, and S. Muresan. Figurative language in recognizing textual entailment. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJCNLP 2021 of Findin…
  20. J. Chen and G. Durrett. Understanding dataset design choices for multi-hop reasoning. In NAACL-HLT (1), pages 4026–4032. Association for Computational Linguistics, 2019.
  21. T. Chen, Z. Jiang, A. Poliak, K. Sakaguchi, and B. V. Durme. Uncertain natural language inference. In D. Jurafsky, J. Chai, N. Schluter, and J. R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 877…
  22. W. Chen, H. Wang, J. Chen, Y. Zhang, H. Wang, S. Li, X. Zhou, and W. Y. Wang. Tabfact: A large-scale dataset for table-based fact verification. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
  23. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, E. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, S. Narang, G. Mishra, A. Yu, V. Y. Zhao, Y. Huang, A. M. Dai, H. Yu, S. Petrov, E. H. Chi, J. Dean, J. Devlin, A. Roberts, D. Zhou, Q. V.… find
  24. P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try arc, the AI2 reasoning challenge. CoRR, abs/1803.05457, 2018.
  25. P. Clark, O. Tafjord, and K. Richardson. Transformers as soft reasoners over language. In IJCAI, pages 3882–3890. ijcai.org, 2020.
  26. A. Conneau, R. Rinott, G. Lample, A. Williams, S. R. Bowman, H. Schwenk, and V. Stoyanov. XNLI: evaluating cross-lingual sentence representations. In E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,… find
  27. A. Creswell and M. Shanahan. Faithful reasoning using large language models. CoRR, abs/2208.14271, 2022.
  28. A. Creswell, M. Shanahan, and I. Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. CoRR, abs/2205.09712, 2022. find
  29. I. Dagan, D. Roth, M. Sammons, and F. M. Zanzotto. Recognizing Textual Entailment: Models and Applications. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2013. find
  30. D. Dai, Y. Sun, L. Dong, Y. Hao, Z. Sui, and F. Wei. Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers. CoRR, abs/2212.10559, 2022.
  31. B. Dalvi, P. Jansen, O. Tafjord, Z. Xie, H. Smith, L. Pipatanangkura, and P. Clark. Explaining answers with entailment trees. In EMNLP (1), pages 7358–7370. Association for Computational Linguistics, 2021.
  32. I. Dasgupta, A. K. Lampinen, S. C. Y. Chan, A. Creswell, D. Kumaran, J. L. McClelland, and F. Hill. Language models show human-like content effects on reasoning. CoRR, abs/2207.07051, 2022.
  33. X. Deng, Y. Su, A. Lees, Y. Wu, C. Yu, and H. Sun. Reasonbert: Pre-trained to reason with distant supervision. In M. Moens, X. Huang, L. Specia, and S. W. Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dom… find
  34. J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In J. Burstein, C. Doran, and T. Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human…
  35. Q. Dong, Z. Qin, H. Xia, T. Feng, S. Tong, H. Meng, L. Xu, Z. Wei, W. Zhan, B. Chang, S. Li, T. Liu, and Z. Sui. Premise-based multimodal reasoning: Conditional inference on joint textual and visual clues. In S. Muresan, P. Nakov, and A. Villavicencio, editors, Proceedings of the 60th Annual Meeting… find
  36. L. Du, X. Ding, T. Liu, and B. Qin. Learning event graph knowledge for abductive reasoning. In ACL/IJCNLP (1), pages 5181–5190. Association for Computational Linguistics, 2021. find
  37. L. Du, X. Ding, K. Xiong, T. Liu, and B. Qin. e-care: a new dataset for exploring explainable causal reasoning. In ACL (1), pages 432–446. Association for Computational Linguistics, 2022. find
  38. D. Emelin, R. L. Bras, J. D. Hwang, M. Forbes, and Y. Choi. Moral stories: Situated reasoning about norms, intents, actions, and their consequences. In EMNLP (1), pages 698–718. Association for Computational Linguistics, 2021. find
  39. Z. Fei, Q. Zhang, T. Gui, D. Liang, S. Wang, W. Wu, and X. Huang. CQG: A simple and effective controlled generation framework for multi-hop question generation. In ACL (1), pages 6896–6906. Association for Computational Linguistics, 2022. find
  40. Y. Feng, X. Chen, B. Y. Lin, P. Wang, J. Yan, and X. Ren. Scalable multi-hop relational reasoning for knowledge-aware question answering. In EMNLP (1), pages 1295–1309. Association for Computational Linguistics, 2020.
  41. M. A. Finocchiaro. Informal logic and the theory of reasoning. Informal Logic, 6(2), 1984. find
  42. M. Forbes, J. D. Hwang, V. Shwartz, M. Sap, and Y. Choi. Social chemistry 101: Learning to reason about social and moral norms. In EMNLP (1), pages 653–670. Association for Computational Linguistics, 2020. find
  43. A.-V. P. Francesco Bellucci. Peirce’s logic. The Internet Encyclopedia of Philosophy, ISSN 2161-0002, 2022. https://iep.utm.edu/peir-log/. ↗
  44. S. Gabriel, S. Hallinan, M. Sap, P. Nguyen, F. Roesner, E. Choi, and Y. Choi. Misinfo reaction frames: Reasoning about readers’ reactions to news headlines. In ACL (1), pages 3108–3127. Association for Computational Linguistics, 2022. find
  45. M. Geva, D. Khashabi, E. Segal, T. Khot, D. Roth, and J. Berant. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. Trans. Assoc. Comput. Linguistics, 9:346–361, 2021. find
  46. A. I. Goldman. Epistemology and cognition. harvard university Press, 1986. find
  47. T. Govier. Critical thinking as argument analysis. Argumentation, 3(2):115–126, 1989. find
  48. I. Habernal, H. Wachsmuth, I. Gurevych, and B. Stein. The argument reasoning comprehension task: Identification and reconstruction of implicit warrants. In NAACL-HLT, pages 1930–1940. Association for Computational Linguistics, 2018.
  49. S. Han, H. Schoelkopf, Y. Zhao, Z. Qi, M. Riddell, L. Benson, L. Sun, E. Zubova, Y. Qiao, M. Burtell, D. Peng, J. Fan, Y. Liu, B. Wong, M. Sailor, A. Ni, L. Nan, J. Kasai, T. Yu, R. Zhang, S. Joty, A. R. Fabbri, W. Kryscinski, X. V. Lin, C. Xiong, and D. Radev. Folio: Natural language reasoning with… — FOLIO: Natural Language Reasoning with First-Order Logic
  50. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  51. J. J. Hintikka. logic. Encyclopedia Britannica, 9 Jun. 2022, 2022. https://www.britannica.com/topic/logic. ↗
  52. M. Ho, A. Sharma, J. Chang, M. Saxon, S. Levy, Y. Lu, and W. Y. Wang. Wikiwhy: Answering and explaining cause-and-effect questions. CoRR, abs/2210.12152, 2022. — [arXiv:2210.12152](https://arxiv.org/abs/2210.12152) find
  53. N. Ho, L. Schmid, and S. Yun. Large language models are reasoning teachers. CoRR, abs/2212.10071, 2022.
  54. X. Ho, A. D. Nguyen, S. Sugawara, and A. Aizawa. Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps. In COLING, pages 6609–6625. International Committee on Computational Linguistics, 2020.
  55. J. Hoffart, M. A. Yosef, I. Bordino, H. Fürstenau, M. Pinkal, M. Spaniol, B. Taneva, S. Thater, and G. Weikum. Robust disambiguation of named entities in text. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Confer… find
  56. R. Hong, H. Zhang, X. Yu, and C. Zhang. METGEN: A module-based entailment tree generation framework for answer explanation. In M. Carpuat, M. de Marneffe, and I. V. M. Ruíz, editors, Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, WA, United States, July 10-15, 2022,… find
  57. M. M. Hossain, V. Kovatchev, P. Dutta, T. Kao, E. Wei, and E. Blanco. An analysis of natural language inference benchmarks through the lens of negation. In B. Webber, T. Cohn, Y. He, and Y. Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 20…
  58. J. Huang and K. C. Chang. Towards reasoning in large language models: A survey. CoRR, abs/2212.10403, 2022.
  59. J. Huang, S. S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han. Large language models can self-improve. CoRR, abs/2210.11610, 2022.
  60. L. Huang, R. L. Bras, C. Bhagavatula, and Y. Choi. Cosmos QA: machine reading comprehension with contextual commonsense reasoning. In EMNLP/IJCNLP (1), pages 2391–2401. Association for Computational Linguistics, 2019.
  61. Y. Huang and M. Yang. Breadth first reasoning graph for multi-hop question answering. In NAACL-HLT, pages 5810–5821. Association for Computational Linguistics, 2021. find
  62. Y. Huang, H. Zhang, R. Hong, X. Liang, C. Zhang, and D. Yu. Metalogic: Logical reasoning explanations with fine-grained structure. CoRR, abs/2210.12487, 2022. , Vol. 1, No. 1, Article . Publication date: May 2023. find
  63. P. J. Hurley. A concise introduction to logic. Cengage Learning, 2014. find
  64. J. D. Hwang, C. Bhagavatula, R. L. Bras, J. Da, K. Sakaguchi, A. Bosselut, and Y. Choi. (comet-) atomic 2020: On symbolic and neural commonsense knowledge graphs. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial I…
  65. N. Inoue, P. Stenetorp, and K. Inui. R4C: A benchmark for evaluating RC systems to get the right answer for the right reason. In ACL, pages 6740–6750. Association for Computational Linguistics, 2020. find
  66. N. Inoue, H. Trivedi, S. Sinha, N. Balasubramanian, and K. Inui. Summarize-then-answer: Generating concise explanations for multi-hop reading comprehension. In EMNLP (1), pages 6064–6080. Association for Computational Linguistics, 2021. find
  67. H. Jhamtani and P. Clark. Learning to explain: Datasets and models for identifying valid reasoning chains in multihop question-answering. In EMNLP (1), pages 137–150. Association for Computational Linguistics, 2020. find
  68. Y. Jiang and M. Bansal. Avoiding reasoning shortcuts: Adversarial evaluation, training, and model development for multi-hop QA. In ACL (1), pages 2726–2736. Association for Computational Linguistics, 2019.
  69. F. Jiao, Y. Guo, X. Song, and L. Nie. Merit: Meta-path guided contrastive learning for logical reasoning. In S. Muresan, P. Nakov, and A. Villavicencio, editors, Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3496–3509. Association for Co… find
  70. J. Jung, L. Qin, S. Welleck, F. Brahman, C. Bhagavatula, R. L. Bras, and Y. Choi. Maieutic prompting: Logically consistent reasoning with recursive explanations. CoRR, abs/2205.11822, 2022.
  71. D. Kahneman. Thinking, fast and slow. Macmillan, 2011. find
  72. S. M. Kazemi, N. Kim, D. Bhatia, X. Xu, and D. Ramachandran. LAMBADA: backward chaining for automated reasoning in natural language. CoRR, abs/2212.13894, 2022.
  73. T. Khot, P. Clark, M. Guerquin, P. Jansen, and A. Sabharwal. QASC: A dataset for question answering via sentence composition. In AAAI, pages 8082–8090. AAAI Press, 2020. find
  74. T. Khot, D. Khashabi, K. Richardson, P. Clark, and A. Sabharwal. Text modular networks: Learning to decompose tasks in the language of existing models. In K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tür, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, and Y. Zhou, editors, Proceedin…
  75. T. Khot, A. Sabharwal, and P. Clark. Scitail: A textual entailment dataset from science question answering. In S. A. McIlraith and K. Q. Weinberger, editors, Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelli… find
  76. T. Klein and M. Nabi. Attention is (not) all you need for commonsense reasoning. In ACL (1), pages 4831–4836. Association for Computational Linguistics, 2019. find
  77. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero-shot reasoners. CoRR, abs/2205.11916, 2022.
  78. T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. P. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, K. Toutanova, L. Jones, M. Kelcey, M. Chang, A. M. Dai, J. Uszkoreit, Q. Le, and S. Petrov. Natural questions: a benchmark for question answering research. Trans. Assoc. Comp…
  79. Y. K. Lal, N. Chambers, R. J. Mooney, and N. Balasubramanian. Tellmewhy: A dataset for answering why-questions in narratives. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJC…
  80. H. Le, C. Sankar, S. Moon, A. Beirami, A. Geramifard, and S. Kottur. DVD: A diagnostic dataset for multi-step reasoning in video grounded dialogue. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th… find
  81. K. Lee, S. Hwang, S. Han, and D. Lee. Robustifying multi-hop QA through pseudo-evidentiality training. In ACL/IJCNLP (1), pages 6110–6119. Association for Computational Linguistics, 2021.
  82. D. B. Lenat. CYC: A large-scale investment in knowledge infrastructure. Commun. ACM, 38(11):32–38, 1995. find
  83. Z. Liang, S. Bethard, and M. Surdeanu. Explainable multi-hop verbal reasoning through internal monologue. In NAACL-HLT, pages 1225–1250. Association for Computational Linguistics, 2021. find
  84. B. Y. Lin, H. Sun, B. Dhingra, M. Zaheer, X. Ren, and W. W. Cohen. Differentiable open-ended commonsense reasoning. In NAACL-HLT, pages 4611–4625. Association for Computational Linguistics, 2021. find
  85. B. Y. Lin, W. Zhou, M. Shen, P. Zhou, C. Bhagavatula, Y. Choi, and X. Ren. Commongen: A constrained text generation challenge for generative commonsense reasoning. In T. Cohn, Y. He, and Y. Liu, editors, Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 Novem… find
  86. K. Lin, O. Tafjord, P. Clark, and M. Gardner. Reasoning over paragraph effects in situations. In A. Fisch, A. Talmor, R. Jia, M. Seo, E. Choi, and D. Chen, editors, Proceedings of the 2nd Workshop on Machine Reading for Question Answering, MRQA@EMNLP 2019, Hong Kong, China, November 4, 2019, pages 5…
  87. H. Liu, L. Cui, J. Liu, and Y. Zhang. Natural language inference in context - investigating contextual reasoning over long texts. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleve… find
  88. H. Liu and P. Singh. Conceptnet—a practical commonsense reasoning tool-kit. BT technology journal, 22(4):211–226, 2004. find
  89. J. Liu, L. Cui, H. Liu, D. Huang, Y. Wang, and Y. Zhang. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. In C. Bessiere, editor, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pages 3622–3628. ijcai.org, 20…
  90. J. Liu, A. Liu, X. Lu, S. Welleck, P. West, R. L. Bras, Y. Choi, and H. Hajishirzi. Generated knowledge prompting for commonsense reasoning. In ACL (1), pages 3154–3169. Association for Computational Linguistics, 2022. find
  91. J. Locke. An essay concerning human understanding. Kay & Troutman, 1847. find
  92. A. Madaan, D. Rajagopal, N. Tandon, Y. Yang, and E. H. Hovy. Could you give me a hint ? generating inference graphs for defeasible reasoning. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021… find
  93. A. Madaan, N. Tandon, D. Rajagopal, P. Clark, Y. Yang, and E. H. Hovy. Think about it! improving defeasible reasoning by first modeling the question scenario. In EMNLP (1), pages 6291–6310. Association for Computational Linguistics, 2021. find
  94. L. C. Magister, J. Mallinson, J. Adámek, E. Malmi, and A. Severyn. Teaching small language models to reason. CoRR, abs/2212.08410, 2022.
  95. T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal. Can a suit of armor conduct electricity? A new dataset for open book question answering. In E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels… find
  96. S. Min, E. Wallace, S. Singh, M. Gardner, H. Hajishirzi, and L. Zettlemoyer. Compositional questions do not necessitate multi-hop reasoning. In ACL (1), pages 4249–4257. Association for Computational Linguistics, 2019.
  97. S. Min, V. Zhong, L. Zettlemoyer, and H. Hajishirzi. Multi-hop reading comprehension through question decomposition and rescoring. In ACL (1), pages 6097–6109. Association for Computational Linguistics, 2019.
  98. N. Mostafazadeh, N. Chambers, X. He, D. Parikh, D. Batra, L. Vanderwende, P. Kohli, and J. F. Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. In K. Knight, A. Nenkova, and O. Rambow, editors, NAACL HLT 2016, The 2016 Conference of the North American Chapter of t… find
  99. Y. Onoe, M. J. Q. Zhang, E. Choi, and G. Durrett. CREAK: A dataset for commonsense reasoning over entity knowledge. In NeurIPS Datasets and Benchmarks, 2021. find
  100. S. Ontañón, J. Ainslie, V. Cvicek, and Z. Fisher. Logicinference: A new dataset for teaching logical inference to seq2seq models. CoRR, abs/2203.15099, 2022. — [arXiv:2203.15099](https://arxiv.org/abs/2203.15099) find
  101. L. Pan, W. Chen, W. Xiong, M. Kan, and W. Y. Wang. Unsupervised multi-hop question answering by question generation. In NAACL-HLT, pages 5866–5880. Association for Computational Linguistics, 2021. find
  102. P. Patel, S. Mishra, M. Parmar, and C. Baral. Is a question decomposition unit all we need? CoRR, abs/2205.12538, 2022. find
  103. C. S. Peirce. Reasoning and the logic of things: The Cambridge conferences lectures of 1898. Harvard University Press, 1992. find
  104. E. Perez, P. S. H. Lewis, W. Yih, K. Cho, and D. Kiela. Unsupervised question decomposition for question answering. In B. Webber, T. Cohn, Y. He, and Y. Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pag…
  105. X. Pi, Q. Liu, B. Chen, M. Ziyadi, Z. Lin, Y. Gao, Q. Fu, J. Lou, and W. Chen. Reasoning like program executors. CoRR, abs/2201.11473, 2022. find
  106. A. Poliak, J. Naradowsky, A. Haldar, R. Rudinger, and B. V. Durme. Hypothesis only baselines in natural language inference. In M. Nissim, J. Berant, and A. Lenci, editors, Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2018, New Orleans, Louisiana,…
  107. O. Press, M. Zhang, S. Min, L. Schmidt, N. A. Smith, and M. Lewis. Measuring and narrowing the compositionality gap in language models. CoRR, abs/2210.03350, 2022. find
  108. B. Prystawski and N. D. Goodman. Why think step-by-step? reasoning emerges from the locality of experience. CoRR, abs/2304.03843, 2023. , Vol. 1, No. 1, Article . Publication date: May 2023. find
  109. P. Qi, H. Lee, T. Sido, and C. D. Manning. Answering open-domain questions of varying reasoning steps from text. In EMNLP (1), pages 3599–3614. Association for Computational Linguistics, 2021. find
  110. S. Qiao, Y. Ou, N. Zhang, X. Chen, Y. Yao, S. Deng, C. Tan, F. Huang, and H. Chen. Reasoning with language model prompting: A survey. CoRR, abs/2212.09597, 2022.
  111. L. Qin, A. Bosselut, A. Holtzman, C. Bhagavatula, E. Clark, and Y. Choi. Counterfactual story reasoning and generation. In EMNLP/IJCNLP (1), pages 5042–5052. Association for Computational Linguistics, 2019. find
  112. L. Qin, V. Shwartz, P. West, C. Bhagavatula, J. D. Hwang, R. L. Bras, A. Bosselut, and Y. Choi. Back to the future: Unsupervised backprop-based decoding for counterfactual and abductive commonsense reasoning. In EMNLP (1), pages 794–805. Association for Computational Linguistics, 2020.
  113. L. Qiu, Y. Xiao, Y. Qu, H. Zhou, L. Li, W. Zhang, and Y. Yu. Dynamically fused graph network for multi-hop reasoning. In ACL (1), pages 6140–6150. Association for Computational Linguistics, 2019.
  114. H. Qu, Y. Cao, J. Gao, L. Ding, and R. Xu. Interpretable proof generation via iterative backward reasoning. In NAACL-HLT, pages 2968–2981. Association for Computational Linguistics, 2022. find
  115. A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever. Improving language understanding by generative pre-training. OpenAI, 2018. find
  116. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1–140:67, 2020.
  117. N. F. Rajani, B. McCann, C. Xiong, and R. Socher. Explain yourself! leveraging language models for commonsense reasoning. In ACL (1), pages 4932–4942. Association for Computational Linguistics, 2019.
  118. H. Rashkin, M. Sap, E. Allaway, N. A. Smith, and Y. Choi. Event2mind: Commonsense inference on events, intents, and reactions. In I. Gurevych and Y. Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 20… find
  119. A. Ravichander, M. Gardner, and A. Marasovic. CONDAQA: A contrastive reading comprehension dataset for reasoning about negation. In Y. Goldberg, Z. Kozareva, and Y. Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United A… find
  120. D. N. Ribeiro, S. Wang, X. Ma, R. Dong, X. Wei, H. Zhu, X. Chen, P. Xu, Z. Huang, A. O. Arnold, and D. Roth. Entailment tree explanations via iterative retrieval-generation reasoner. In M. Carpuat, M. de Marneffe, and I. V. M. Ruíz, editors, Findings of the Association for Computational Linguistics:… find
  121. M. Roemmele, C. A. Bejan, and A. S. Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In Logical Formalizations of Commonsense Reasoning, Papers from the 2011 AAAI Spring Symposium, Technical Report SS-11-06, Stanford, California, USA, March 21-23, 2011. AAAI,… find
  122. R. Rudinger, V. Shwartz, J. D. Hwang, C. Bhagavatula, M. Forbes, R. L. Bras, N. A. Smith, and Y. Choi. Thinking like a skeptic: Defeasible inference in natural language. In T. Cohn, Y. He, and Y. Liu, editors, Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20… find
  123. D. D. Runes. The dictionary of philosophy. Citadel Press, 2001. find
  124. M. Sadat and C. Caragea. Scinli: A corpus for natural language inference on scientific text. In S. Muresan, P. Nakov, and A. Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27,…
  125. M. Saeidi, M. Bartolo, P. S. H. Lewis, S. Singh, T. Rocktäschel, M. Sheldon, G. Bouchard, and S. Riedel. Interpretation of natural language rules in conversational machine reading. In E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, editors, Proceedings of the 2018 Conference on Empirical Method…
  126. S. Saha, S. Ghosh, S. Srivastava, and M. Bansal. Prover: Proof generation for interpretable reasoning over rules. In EMNLP (1), pages 122–136. Association for Computational Linguistics, 2020.
  127. S. Saha, Y. Nie, and M. Bansal. Conjnli: Natural language inference over conjunctive sentences. In B. Webber, T. Cohn, Y. He, and Y. Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 8240–8252. Associ…
  128. S. Saha, P. Yadav, and M. Bansal. multiprover: Generating multiple proofs for improved interpretability in rule reasoning. In NAACL-HLT, pages 3662–3677. Association for Computational Linguistics, 2021. find
  129. S. Sanyal, Z. Liao, and X. Ren. Robustlr: Evaluating robustness to logical perturbation in deductive reasoning. CoRR, abs/2205.12598, 2022. — [arXiv:2205.12598](https://arxiv.org/abs/2205.12598) find
  130. S. Sanyal, H. Singh, and X. Ren. Fairr: Faithful and robust deductive reasoning over natural language. In ACL (1), pages 1075–1093. Association for Computational Linguistics, 2022. Natural Language Reasoning, A Survey • 33 , Vol. 1, No. 1, Article . Publication date: May 2023. • Fei Yu, Hongbo Zhang… find
  131. S. Sanyal, Y. Xu, S. Wang, Z. Yang, R. Pryzant, W. Yu, C. Zhu, and X. Ren. APOLLO: A simple approach for adaptive pretraining of language models for logical reasoning. CoRR, abs/2212.09282, 2022. find
  132. M. Sap, R. L. Bras, E. Allaway, C. Bhagavatula, N. Lourie, H. Rashkin, B. Roof, N. A. Smith, and Y. Choi. ATOMIC: an atlas of machine commonsense for if-then reasoning. In AAAI, pages 3027–3035. AAAI Press, 2019. find
  133. M. Sap, S. Gabriel, L. Qin, D. Jurafsky, N. A. Smith, and Y. Choi. Social bias frames: Reasoning about social and power implications of language. In ACL, pages 5477–5490. Association for Computational Linguistics, 2020.
  134. M. Sap, H. Rashkin, D. Chen, R. L. Bras, and Y. Choi. Social iqa: Commonsense reasoning about social interactions. In EMNLP/IJCNLP (1), pages 4462–4472. Association for Computational Linguistics, 2019. find
  135. A. Saparov and H. He. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. CoRR, abs/2210.01240, 2022.
  136. K. Shridhar, A. Stolfo, and M. Sachan. Distilling multi-step reasoning capabilities of large language models into smaller models via semantic decompositions. CoRR, abs/2212.00193, 2022.
  137. K. Sinha, S. Sodhani, J. Dong, J. Pineau, and W. L. Hamilton. CLUTRR: A diagnostic benchmark for inductive reasoning from text. In EMNLP/IJCNLP (1), pages 4505–4514. Association for Computational Linguistics, 2019. find
  138. R. Speer, J. Chin, and C. Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In S. Singh and S. Markovitch, editors, Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA, pages 4444–4451. AAAI Press, 2017… find
  139. A. Srivastava, A. Rastogi, A. Rao, A. A. M. Shoeb, A. Abid, A. Fisch, A. R. Brown, A. Santoro, A. Gupta, A. Garriga-Alonso, A. Kluska, A. Lewkowycz, A. Agarwal, A. Power, A. Ray, A. Warstadt, A. W. Kocurek, A. Safaya, A. Tazarv, A. Xiang, A. Parrish, A. Nie, A. Hussain, A. Askell, A. Dsouza, A. Raha… — Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
  140. A. Suhr, S. Zhou, A. Zhang, I. Zhang, H. Bai, and Y. Artzi. A corpus for reasoning about natural language grounded in photographs. In A. Korhonen, D. R. Traum, and L. Màrquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, Jul…
  141. M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, and J. Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. CoRR, abs/2210.09261, 2022.
  142. O. Tafjord, B. Dalvi, and P. Clark. Proofwriter: Generating implications, proofs, and abductive statements over natural language. In ACL/IJCNLP (Findings), volume ACL/IJCNLP 2021 of Findings of ACL, pages 3621–3634. Association for Computational Linguistics, 2021.
  143. O. Tafjord, B. D. Mishra, and P. Clark. Entailer: Answering questions with faithful and truthful chains of reasoning. CoRR, abs/2210.12217, 2022.
  144. A. Talmor, J. Herzig, N. Lourie, and J. Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In J. Burstein, C. Doran, and T. Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human L…
  145. A. Talmor, O. Tafjord, P. Clark, Y. Goldberg, and J. Berant. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge. In NeurIPS, 2020.
  146. A. Talmor, O. Yoran, R. L. Bras, C. Bhagavatula, Y. Goldberg, Y. Choi, and J. Berant. Commonsenseqa 2.0: Exposing the limits of AI through gamification. In J. Vanschoren and S. Yeung, editors, Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datase…
  147. A. Tamborrino, N. Pellicanò, B. Pannier, P. Voitot, and L. Naudin. Pre-training is (almost) all you need: An application to commonsense reasoning. In ACL, pages 3878–3887. Association for Computational Linguistics, 2020. find
  148. N. Tandon, B. Dalvi, J. Grus, W. Yih, A. Bosselut, and P. Clark. Reasoning about actions and state changes by injecting commonsense knowledge. In E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brus… find
  149. N. Tandon, B. Dalvi, K. Sakaguchi, P. Clark, and A. Bosselut. WIQA: A dataset for "what if..." reasoning over procedural text. In EMNLP/IJCNLP (1), pages 6075–6084. Association for Computational Linguistics, 2019. find
  150. H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal. Is multihop QA in dire condition? measuring and reducing disconnected reasoning. In EMNLP (1), pages 8846–8863. Association for Computational Linguistics, 2020. find
  151. H. Trivedi, N. Balasubramanian, T. Khot, and A. Sabharwal. Musique: Multihop questions via single-hop question composition. Trans. Assoc. Comput. Linguistics, 10:539–554, 2022. , Vol. 1, No. 1, Article . Publication date: May 2023. — ♫ MuSiQue: Multihop Questions via Single-hop Question Composition
  152. M. Tsuchiya. Performance impact caused by hidden bias of training data for recognizing textual entailment. In N. Calzolari, K. Choukri, C. Cieri, T. Declerck, S. Goggi, K. Hasida, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, S. Piperidis, and T. Tokunaga, editors, Proceedings o…
  153. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Ann…
  154. D. Vilares and C. Gómez-Rodríguez. HEAD-QA: A healthcare dataset for complex reasoning. In ACL (1), pages 960–966. Association for Computational Linguistics, 2019.
  155. D. N. Walton. What is reasoning? what is an argument? The journal of Philosophy, 87(8):399–419, 1990. find
  156. X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, and D. Zhou. Self-consistency improves chain of thought reasoning in language models. CoRR, abs/2203.11171, 2022.
  157. J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, and W. Fedus. Emergent abilities of large language models. CoRR, abs/2206.07682, 2022.
  158. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. H. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. CoRR, abs/2201.11903, 2022.
  159. J. Welbl, P. Stenetorp, and S. Riedel. Constructing datasets for multi-hop reading comprehension across documents. Trans. Assoc. Comput. Linguistics, 6:287–302, 2018.
  160. J. Weston, A. Bordes, S. Chopra, and T. Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. In ICLR (Poster), 2016.
  161. A. Williams, N. Nangia, and S. R. Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In M. A. Walker, H. Ji, and A. Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T…
  162. Y. Wu, M. Gardner, P. Stenetorp, and P. Dasigi. Generating data to mitigate spurious correlations in natural language inference datasets. In S. Muresan, P. Nakov, and A. Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa…
  163. S. M. Xie, A. Raghunathan, P. Liang, and T. Ma. An explanation of in-context learning as implicit bayesian inference. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
  164. H. Yanaka, K. Mineshima, D. Bekki, K. Inui, S. Sekine, L. Abzianidze, and J. Bos. Can neural networks understand monotonicity reasoning? In T. Linzen, G. Chrupala, Y. Belinkov, and D. Hupkes, editors, Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NL… find
  165. H. Yanaka, K. Mineshima, D. Bekki, K. Inui, S. Sekine, L. Abzianidze, and J. Bos. HELP: A dataset for identifying shortcomings of neural models in monotonicity reasoning. In R. Mihalcea, E. Shutova, L. Ku, K. Evang, and S. Poria, editors, Proceedings of the Eighth Joint Conference on Lexical and Com… find
  166. K. Yang, J. Deng, and D. Chen. Generating natural language proofs with verifier-guided search. CoRR, abs/2205.12443, 2022. find
  167. Z. Yang, L. Dong, X. Du, H. Cheng, E. Cambria, X. Liu, J. Gao, and F. Wei. Language models as inductive reasoners. CoRR, abs/2212.10923, 2022.
  168. Z. Yang, X. Du, R. Mao, J. Ni, and E. Cambria. Logical reasoning over natural language as knowledge representation: A survey. CoRR, abs/2303.12023, 2023.
  169. Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In EMNLP, pages 2369–2380. Association for Computational Linguistics, 2018.
  170. M. Yasunaga, H. Ren, A. Bosselut, P. Liang, and J. Leskovec. QA-GNN: reasoning with language models and knowledge graphs for question answering. In NAACL-HLT, pages 535–546. Association for Computational Linguistics, 2021. find
  171. X. Ye, S. Iyer, A. Celikyilmaz, V. Stoyanov, G. Durrett, and R. Pasunuru. Complementary explanations for effective in-context learning. CoRR, abs/2211.13892, 2022.
  172. D. Yin, L. H. Li, Z. Hu, N. Peng, and K. Chang. Broaden the vision: Geo-diverse visual commonsense reasoning. In M. Moens, X. Huang, L. Specia, and S. W. Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Domi… find
  173. W. Yin, D. R. Radev, and C. Xiong. Docnli: A large-scale dataset for document-level natural language inference. In C. Zong, F. Xia, W. Li, and R. Navigli, editors, Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, Natural Language Reasoning,…
  174. N. Young, Q. Bao, J. Bensemann, and M. Witbrock. Abductionrules: Training transformers to explain unexpected inputs. In S. Muresan, P. Nakov, and A. Villavicencio, editors, Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 218–227. Associati… find
  175. J. Yu, W. Liu, S. Qiu, Q. Su, K. Wang, X. Quan, and J. Yin. Low-resource generation of multi-hop reasoning questions. In ACL, pages 6729–6739. Association for Computational Linguistics, 2020. find
  176. P. Yu, T. Wang, O. Golovneva, B. AlKhamissy, G. Ghosh, M. T. Diab, and A. Celikyilmaz. ALERT: adapting language models to reasoning tasks. CoRR, abs/2212.08286, 2022. — [arXiv:2212.08286](https://arxiv.org/abs/2212.08286) find
  177. W. Yu, Z. Jiang, Y. Dong, and J. Feng. Reclor: A reading comprehension dataset requiring logical reasoning. In ICLR. OpenReview.net, 2020.
  178. E. Zelikman, Y. Wu, J. Mu, and N. D. Goodman. Star: Bootstrapping reasoning with reasoning. CoRR, abs/2203.14465, 2022.
  179. R. Zellers, Y. Bisk, R. Schwartz, and Y. Choi. SWAG: A large-scale adversarial dataset for grounded commonsense inference. In E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, Octob… find
  180. R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi. Hellaswag: Can a machine really finish your sentence? In A. Korhonen, D. R. Traum, and L. Màrquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 20… find
  181. H. Zhang, L. H. Li, T. Meng, K. Chang, and G. V. den Broeck. On the paradox of learning to reason from data. CoRR, abs/2205.11502, 2022. — [arXiv:2205.11502](https://arxiv.org/abs/2205.11502) find
  182. L. Zhang, Q. Lyu, and C. Callison-Burch. Reasoning about goals, steps, and temporal ordering with wikihow. In EMNLP (1), pages 4630–4639. Association for Computational Linguistics, 2020. find
  183. X. Zhang, A. Bosselut, M. Yasunaga, H. Ren, P. Liang, C. D. Manning, and J. Leskovec. Greaselm: Graph reasoning enhanced language models. In ICLR. OpenReview.net, 2022. find
  184. Z. Zhang, A. Zhang, M. Li, and A. Smola. Automatic chain of thought prompting in large language models. CoRR, abs/2210.03493, 2022.
  185. C. Zhao, C. Xiong, C. Rosset, X. Song, P. N. Bennett, and S. Tiwary. Transformer-xh: Multi-evidence reasoning with extra hop attention. In ICLR. OpenReview.net, 2020. find
  186. C. Zheng and P. Kordjamshidi. SRLGRN: semantic role labeling graph reasoning network. In EMNLP (1), pages 8881–8891. Association for Computational Linguistics, 2020. find
  187. V. Zhong and L. Zettlemoyer. E3: entailment-driven extracting and editing for conversational machine reading. In A. Korhonen, D. R. Traum, and L. Màrquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019,… find
  188. W. Zhong, T. Ma, J. Wang, J. Yin, T. Zhao, C.-Y. Lin, and N. Duan. Disentangling reasoning capabilities from language models with compositional reasoning transformers. CoRR, abs/2210.11265, 2022. find
  189. W. Zhong, S. Wang, D. Tang, Z. Xu, D. Guo, Y. Chen, J. Wang, J. Yin, M. Zhou, and N. Duan. Analytical reasoning of text. In M. Carpuat, M. de Marneffe, and I. V. M. Ruíz, editors, Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pag… find
  190. B. Zhou, K. Richardson, X. Yu, and D. Roth. Learning to decompose: Hypothetical question decomposition based on comparable texts. CoRR, abs/2210.16865, 2022. find
  191. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le, and E. Chi. Least-to-most prompting enables complex reasoning in large language models. CoRR, abs/2205.10625, 2022.
  192. P. Zhou, R. Khanna, S. Lee, B. Y. Lin, D. Ho, J. Pujara, and X. Ren. RICA: evaluating robust inference capabilities based on commonsense axioms. In M. Moens, X. Huang, L. Specia, and S. W. Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 202… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yu-2023-natural-language-reasoning-survey/