Elamur Research⌕
☰
home / papers / 2022 / Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning

Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning

Paper · 2022 · Oyvind Tafjord, Bhavana Dalvi Mishra and Peter Clark

Summary

Language models answer questions well but rarely show how an answer derives from what they actually "know." Explanations they produce may not entail the answer (unfaithful) or may contain statements the model would reject (untruthful). Entailer addresses both problems. It turns each answer option into a declarative hypothesis and then builds a proof tree by backward chaining. At every step, a trained model proposes several candidate premise sets. The same model then queries itself to check that it accepts each premise and the entailment step. Only verified steps survive. The tree keeps growing until the root's score stops improving or a depth limit is reached, and the answer with the strongest proof wins. Training uses EntailmentBank plus a new bootstrapped, crowd-annotated set of positive and negative entailments. The frozen model is then applied zero-shot to OBQA and QuaRTz. Answer accuracy matches direct QA, and human judges find most chains convincing. Because the chains expose the model's beliefs, users can locate the faulty belief behind a wrong answer and correct it.

Key points

  • The authors call Entailer the first system to produce multistep reasoning chains that are both faithful and truthful. Here "belief" is defined operationally: the model believes X if it answers "yes" when asked whether X is true.
  • The model is used in three ways: generating premises from a hypothesis (H → P), scoring how true a single statement is (S_d), and scoring whether an entailment step is valid (S_e). To turn question–answer pairs into hypotheses, a separate T5-11B model trained on QA2D is used.
  • A new crowdsourced dataset of multi-premise entailments roughly doubles the data available in EntailmentBank. Unlike EntailmentBank, it also includes negative examples.
  • In human evaluation on OBQA and QuaRTz, users judged that 70%+ of chains clearly showed how the answer follows from the facts. This is well above explanations from the Macaw baseline, and QA accuracy was preserved.
  • The main limitations are that some steps are invalid or near-tautological, entailment is defined imprecisely, and the model's beliefs can be inconsistent. The approach is also costly, at about 360 seconds per 4-way question for proofs up to depth 3. Data and models are released at allenai.org/data/entailer.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Allen Institute for Artificial IntelligenceSeattle · United States304 works · 106,137 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,685 worksQuestion answering · 487 works

Cite

Cite as Oyvind Tafjord, Bhavana Dalvi Mishra and Peter Clark. 2022. Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning. arXiv:2210.12217.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2210.12217 · arXiv 2210.12217

@misc{tafjord2022entailer,
  title = {{Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning}},
  author = {Oyvind Tafjord, Bhavana Dalvi Mishra and Peter Clark},
  year = {2022},
  doi = {10.48550/arxiv.2210.12217},
  eprint = {2210.12217},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2210.12217}
}

References

40 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019. COMET: Commonsense transformers for automatic knowledge graph construction. In ACL.
  2. Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, and Greg Durrett. 2022. Natural language deduction through search over statement compositions. ArXiv, abs/2201.06028. find
  3. Kaj Bostrom, Xinyu Zhao, Swarat Chaudhuri, and Greg Durrett. 2021. Flexible generation of natural language deductions. In EMNLP. find
  4. Tom B. Brown et al. 2020. Language models are few-shot learners. In NeurIPS.
  5. Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020. Transformers as soft reasoners over language. In IJCAI’20.
  6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. ArXiv, abs/2110.14168.
  7. Antonia Creswell, Murray Shanahan, and Irina Higgins. 2022. Selection-inference: Exploiting large language models for interpretable logical reasoning. ArXiv, abs/2205.09712. find
  8. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing Textual Entailment: Models and Applications. Morgan and Claypool. find
  9. Bhavana Dalvi, Peter A. Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021. Explaining answers with entailment trees. In EMNLP.
  10. Dorottya Demszky, Kelvin Guu, and Percy Liang. 2018. Transforming question answering datasets into natural language inference datasets. ArXiv, abs/1809.02922.
  11. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, E. Hovy, H. Schutze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. ArXiv, abs/2102.01017. find
  12. Saadia Gabriel, Chandra Bhagavatula, Vered Shwartz, Ronan Le Bras, Maxwell Forbes, and Yejin Choi. 2021. Paragraph-level commonsense transformers with recurrent memory. In AAAI. find
  13. Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. In ICLR.
  14. Ruixin Hong, Hongming Zhang, Xintong Yu, and Changshui Zhang. 2022. Metgen: A module-based entailment tree generation framework for answer explanation. ArXiv, abs/2205.02593. find
  15. Aditya Kalyanpur, Tom Breloff, and David A. Ferrucci. 2020. Braid: Weaving symbolic and neural knowledge into coherent logical explanations. arXiv: Computation and Language. find
  16. Nora Kassner and H. Schütze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In ACL.
  17. Nora Kassner, Oyvind Tafjord, Hinrich Schutze, and Peter Clark. 2021. BeliefBank: Adding memory to a pre-trained language model for a systematic notion of belief. In EMNLP. find
  18. Tao Li, Vivek Gupta, Maitrey Mehta, and Vivek Srikumar. 2019. A logic-driven framework for consistency of neural models. In EMNLP. find
  19. Yujia Li, David H. Choi, et al. 2022. Competition-level code generation with alphacode. ArXiv, abs/2203.07814.
  20. Christopher D. Manning and Bill MacCartney. 2009. Natural language inference. Stanford University. find
  21. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? A new dataset for open book question answering. In EMNLP. find
  22. Bhavana Dalvi Mishra, Oyvind Tafjord, and Peter Clark. 2022. Towards teachable reasoning systems: Using a dynamic memory of user feedback for continual system improvement. In EMNLP. find
  23. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. 2021. Fast model editing at scale. ArXiv, abs/2110.11309. find
  24. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2021. Show your work: Scratchpads for intermediate computation with language models. ArXiv, abs/2112.00114.
  25. F. Petroni, Tim Rocktäschel, Patrick Lewis, A. Bakhtin, Y. Wu, Alexander H. Miller, and S. Riedel. 2019. Language models as knowledge bases? In EMNLP.
  26. Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henry Zhu, Xinchi Chen, Zhiheng Huang, Peng Xu, Andrew O. Arnold, and Dan Roth. 2022. Entailment tree explanations via iterative retrieval-generation reasoner. ArXiv, abs/2205.09224. find
  27. Marco Tulio Ribeiro, Carlos Guestrin, and Sameer Singh. 2019. Are red roses red? Evaluating consistency of question-answering models. In ACL. find
  28. Eric Schwitzgebel. 2019. Belief. Stanford Encyclopedia of Philosophy. Https://plato.stanford.edu/entries/belief/. find
  29. Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Unsupervised commonsense question answering with self-talk. In EMNLP, pages 4615–4629. find
  30. Oyvind Tafjord and Peter Clark. 2021. General-purpose question-answering with Macaw. ArXiv, abs/2109.02593. find
  31. Oyvind Tafjord, Matt Gardner, Kevin Lin, and Peter Clark. 2019. QuaRTz: An open-domain dataset of qualitative relationship questions. ArXiv, abs/1909.03553.
  32. Oyvind Tafjord, B. D. Mishra, and P. Clark. 2020. ProofWriter: Generating implications, proofs, and abductive statements over natural language. ArXiv, abs/2012.13048.
  33. Alon Talmor, Oyvind Tafjord, P. Clark, Y. Goldberg, and Jonathan Berant. 2020. LeapOfThought: Teaching pre-trained models to systematically reason over implicit knowledge. In NeurIPS. find
  34. Niket Tandon, Aman Madaan, Peter Clark, and Yiming Yang. 2022. Memory-assisted prompt editing to improve GPT-3 after deployment. In ACL Workshop on Commonsense Representation and Reasoning (CSRR’22). (also arxiv:2201.06009). find
  35. Stefano Teso and Kristian Kersting. 2019. Explanatory interactive machine learning. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. find
  36. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Rationale-augmented ensembles in language models. ArXiv, abs/2207.00747.
  37. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201.11903.
  38. Sarah Wiegreffe and Ana Marasović. 2021. Teach me to explain: A review of datasets for explainable NLP. ArXiv, abs/2102.12060. — [arXiv:2102.12060](https://arxiv.org/abs/2102.12060) find
  39. Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, and Peter Jansen. 2020. WorldTree V2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference. In LREC. find
  40. Kaiyu Yang, Jia Deng, and Danqi Chen. 2022. Generating natural language proofs with verifier-guided search. ArXiv, abs/2205.12443. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tafjord-2022-entailer-answering-questions-faithful/