Elamur Research⌕
☰
home / papers / 2023 / Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework

Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework

Summary

Large language models often invent facts. Chain-of-Thought (CoT) prompting makes their reasoning visible, but it does not fix this weakness on knowledge-heavy tasks. The authors propose Verify-and-Edit (VE), a framework built on GPT-3 that repairs reasoning chains after they are generated, rather than only choosing among them.

VE works in stages:

  • Flag uncertain cases. Self-consistency sampling identifies predictions where fewer than ⌈n/2⌉ sampled paths agree. Confident predictions are left unchanged.
  • Verify. For uncertain cases, the model writes questions that check the claims in its rationale.
  • Retrieve. It answers those questions using external evidence from Wikipedia or Google search.
  • Edit and re-answer. It rewrites the rationale with the corrected facts and produces a new answer.

The method is tested on open-domain multi-hop QA (Adversarial HotpotQA, 2WikiMultihop) and fact verification (Fever). VE beats CoT self-consistency and comparable retrieval-augmented methods such as ReAct, while using shorter, conversational prompts. Human judges also rate its reasoning as more factual. The work matters because it offers an inexpensive way to ground LLM reasoning in current external sources, which is valuable wherever factual reliability determines user trust.

Key points

  • On 2WikiMultihop with open-domain search, VE reaches 33.6% accuracy versus 27.7% for CoT Self-Consistency. On Adversarial HotpotQA, it gives roughly 3.8x the accuracy gain of similar retrieval-augmented models.
  • The only hyperparameter is the consistency threshold. An ablation on Adversarial HotpotQA shows performance peaks near ⌈n/2⌉ (3 in their setup). Higher thresholds mostly re-edit correct answers and add noise.
  • In a human study of 50 HotpotQA questions, two annotators rated VE chains higher on factual consistency in 53% of cases, compared with 17% for CoT-SC (Cohen κ = 0.25). The annotators used Google search in every case.
  • VE is cheaper than ReAct: editing one Fever instance via the GPT-3 API costs $0.014, compared with $0.017 for ReAct.
  • The code is released at github.com/RuochenZhao/Verify-and-Edit. The authors note that gains are smaller on simpler or commonsense tasks and depend on how well consistency separates correct from incorrect predictions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

88 papers of our library cite it (of the 140884 whose bibliographies we read); 8 of them have a page here, 80 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.acl-long.320

@inproceedings{zhao2023verify,
  title = {{Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework}},
  author = {Zhao, Ruochen and Li, Xingxuan and Joty, Shafiq and Qin, Chengwei and Bing, Lidong},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.320},
  url = {https://doi.org/10.18653/v1/2023.acl-long.320}
}

References

25 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. find
  2. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879, Vancouver, Canada. Association for Computational Lin… find our card
  3. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311. find
  4. Antonia Creswell, Murray Shanahan, and Irina Higgins. 2022. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712. find
  5. Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2022. Roscoe: A suite of metrics for scoring step-by-step reasoning. arXiv preprint arXiv:2212.07919. find our card
  6. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. Realm: Retrieval-augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org. find
  7. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, Barcelona, Spain (Online). International Comm… find our card
  8. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Onli… find our card
  9. Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. 2022. Internet-augmented language models through few-shot prompting for open-domain question answering. find our card
  10. Gary Marcus. 2022. Is chatgpt really a “code red” for google search? find
  11. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. find
  12. OpenAI-Blog. 2022. Chatgpt: Optimizing language models for dialogue. find
  13. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155. find
  14. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350. find
  15. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… find
  16. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo… find our card
  17. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. find our card
  18. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. find our card
  19. Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2022. Reframing human-AI collaboration for generating free-text explanations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie… find
  20. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369… find our card
  21. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. find our card
  22. Xi Ye and Greg Durrett. 2022. The unreliability of explanations in few-shot prompting for textual reasoning. In Advances in Neural Information Processing Systems. find our card
  23. Ruochen Zhao, Xingxuan Li, Yew Ken Chia, Bosheng Ding, and Lidong Bing. 2023. Can chatgpt-like generative models guarantee factual accuracy? on the mistakes of new generation search engines. arXiv preprint arXiv:2304.11076. — [arXiv:2304.11076](https://arxiv.org/abs/2304.11076) find
  24. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. 2022. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625. find our card
  25. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2023-verify-edit-knowledge-enhanced-chain/