Self-Consistency Improves Chain of Thought Reasoning in Language Models

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Mathematics & statistics

Cited by

2341 papers of our library cite it (of the 140673 whose bibliographies we read); 130 of them have a page here, the first 100 below, 2211 not yet.

• L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning
• Large Language Models for Mathematical Reasoning: Progresses and Challenges
• Least-to-most prompting enables complex reasoning in large language models
• PaLM 2 Technical Report
• Llemma: An Open Language Model For Mathematics
• Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering
• Qwen Technical Report
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
• MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
• TheoremQA: A Theorem-driven Question Answering Dataset
• AlphaMath Almost Zero: Process Supervision without Process
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
• Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Chain-of-Verification Reduces Hallucination in Large Language Models
• Improving Factuality and Reasoning in Language Models through Multiagent Debate
• Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
• Complexity-Based Prompting for Multi-Step Reasoning
• Specializing Smaller Language Models towards Multi-Step Reasoning
• Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation
• Gemini: A Family of Highly Capable Multimodal Models
• ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
• rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
• HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
• FOLIO: Natural Language Reasoning with First-Order Logic
• Token-Budget-Aware LLM Reasoning
• AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators
• Exploring Human-Like Translation Strategy with Large Language Models
• MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
• V-STaR: Training Verifiers for Self-Taught Reasoners
• Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
• C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models
• Large Language Models Can Self-Improve
• Large Language Models Cannot Self-Correct Reasoning Yet
• Towards Reasoning in Large Language Models: A Survey
• A Survey on Large Language Models for Code Generation
• Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
• Challenges and Applications of Large Language Models
• Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
• Language Models can Solve Computer Tasks
• VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
• Large language models are zero-shot reasoners
• Training Language Models to Self-Correct via Reinforcement Learning
• Solving Quantitative Reasoning Problems with Language Models
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society
• Making Language Models Better Reasoners with Step-Aware Verifier
• Multi-step Jailbreaking Privacy Attacks on ChatGPT
• Symbolic Chain-of-Thought Distillation: Small Models Can Also ``Think'' Step-by-Step
• GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers
• LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate
• Let's Verify Step by Step
• Deductive Verification of Chain-of-Thought Reasoning
• AgentBench: Evaluating LLMs as Agents
• Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment
• Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
• MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
• WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct
• Improve Mathematical Reasoning in Language Models by Automated Process Supervision
• Faithful Chain-of-Thought Reasoning
• Query Rewriting in Retrieval-Augmented Large Language Models
• MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems
• Teaching Small Language Models to Reason
• Augmented Language Models: a Survey
• A Comprehensive Overview of Large Language Models
• LEVER: Learning to Verify Language-to-Code Generation with Execution
• Capabilities of GPT-4 on Medical Challenge Problems
• LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers
• Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
• Unifying Large Language Models and Knowledge Graphs: A Roadmap
• REFINER: Reasoning Feedback on Intermediate Representations
• Reasoning with Language Model Prompting: A Survey
• Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
• Tool Learning with Foundation Models
• A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
• Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
• On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
• HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face
• Large Language Models Can Be Easily Distracted by Irrelevant Context
• Distilling Reasoning Capabilities into Smaller Language Models
• Prompting GPT-3 To Be Reliable
• Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
• Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
• Towards Expert-Level Medical Question Answering with Large Language Models
• To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
• Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
• Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
• UL2: Unifying Language Learning Paradigms
• ReFT: Reasoning with Reinforced Fine-Tuning
• Solving math word problems with process- and outcome-based feedback
• MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning
• SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
• Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
• A Survey on Large Language Model based Autonomous Agents
• Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2203.11171 · arXiv 2203.11171

@misc{wang2022self,
  title = {{Self-Consistency Improves Chain of Thought Reasoning in Language Models}},
  author = {Wang, Xuezhi and Wei, Jason and Schuurmans, Dale and Le, Quoc and Chi, Ed and Narang, Sharan and Chowdhery, Aakanksha and Zhou, Denny},
  year = {2022},
  doi = {10.48550/arxiv.2203.11171},
  eprint = {2203.11171},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2203.11171}
}

References

65 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David H. Ackley, Geoffrey E. Hinton, and Terrence J. Sejnowski. A learning algorithm for boltzmann machines. Cognitive Science, 9(1):147–169, 1985. ISSN 0364-0213. URL https://www. sciencedirect.com/science/article/pii/S0364021385800124. find
2. Daniel Adiwardana, Minh-Thang Luong, David R. So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. Towards a human-like open-domain chatbot, 2020. find our card
3. Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. MathQA: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Lin… find our card
4. Daniel Andor, Luheng He, Kenton Lee, and Emily Pitler. Giving BERT a calculator: Finding operations and arguments with reading comprehension. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… find
5. Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. Learning to retrieve reasoning paths over wikipedia graph for question answering. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum? id=SJgVHkrYDH. ↗ our card
6. Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. The second pascal recognising textual entailment challenge. In Proceedings of the second PASCAL challenges workshop on recognising textual entailment, 2006. find
7. Dhruv Batra, Payman Yadollahpour, Abner Guzman-Rivera, and Gregory Shakhnarovich. Diverse m-best solutions in markov random fields. In Proceedings of the 12th European Conference on Computer Vision - Volume Part V, ECCV’12, pp. 1–16, Berlin, Heidelberg, 2012. Springer-Verlag. ISBN 9783642337147. URL… — [doi:10.1007/978-3-642-33715-4\_1](https://doi.org/10.1007/978-3-642-33715-4_1) find
8. Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. The fifth pascal recognizing textual entailment challenge. In TAC, 2009. find
9. BIG-bench collaboration. Beyond the imitation game: Measuring and extrapolating the capabilities of language models. In preparation, 2021. URL https://github.com/google/ BIG-bench/. ↗
10. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… find
11. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. e-snli: Natural language inference with natural language explanations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems 31, pp. 953… find our card
12. Oana-Maria Camburu, Brendan Shillingford, Pasquale Minervini, Thomas Lukasiewicz, and Phil Blunsom. Make up your mind! adversarial generation of inconsistent natural language explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4157–4165, Onli… find
13. Jifan Chen, Shih-Ting Lin, and Greg Durrett. Multi-hop question answering via reasoning chains. CoRR, abs/1910.02610, 2019. URL http://arxiv.org/abs/1910.02610. — [arXiv:1910.02610](https://arxiv.org/abs/1910.02610) ↗
14. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. find our card
15. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… find
16. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In NAACL, 2019. find our card
17. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs/1803.05457, 2018. find
18. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. find our card
19. Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine Learning Challenges Workshop, pp. 177–190. Springer, 2005. find
20. Bryan Eikema and Wilker Aziz. Is MAP decoding all you need? the inadequacy of the mode in neural machine translation. In Proceedings of the 28th International Conference on Computational Linguistics, pp. 4506–4520, Barcelona, Spain (Online), December 2020. International Committee on Computational Li… find
21. Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031, 2021. doi: 10.1162/tacl\_a\_00410. URL htt… find
22. Jonathan St BT Evans. Intuition and reasoning: A dual-process perspective. Psychological Inquiry, 21(4):313–326, 2010. find
23. Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 889–898, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v… find
24. Jessica Ficler and Yoav Goldberg. Controlling linguistic style aspects in neural language generation. In Proceedings of the Workshop on Stylistic Variation, pp. 94–104, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/v1/W17-4912. URL https: //aclantholog… find our card
25. Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 3816… find our card
26. Mor Geva, Ankit Gupta, and Jonathan Berant. Injecting numerical reasoning skills into language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. doi: 10.18653/v1/2020.acl-main.89. URL https://aclanthology.org/2020. acl-main.89. ↗ our card
27. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? A question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 2021. URL https://aclanthology.org/ 2021.tacl-1.21. ↗
28. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. The third pascal recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pp. 1–9. Association for Computational Linguistics, 2007. find
29. Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. Learning to write with cooperative discriminators. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1638–1649, Melbourne, Australia, July 2018… find our card
30. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations, 2020. URL https: //openreview.net/forum?id=rygGQyrFvH. find
31. Mohammad Javad Hosseini, Hannaneh Hajishirzi, Oren Etzioni, and Nate Kushman. Learning to solve arithmetic word problems with verb categorization. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014. doi: 10.3115/v1/D14-1058. URL https://aclantholo… find our card
32. Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. UNIFIEDQA: Crossing format boundaries with a single QA system. In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1896–1907, Online, November 2020. Association… find our card
33. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=e… ↗ our card
34. Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. MAWPS: A math word problem repository. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016. doi: 10.18653/v1/N16-1… find
35. Yihuai Lan, Lei Wang, Qiyuan Zhang, Yunshi Lan, Bing Tian Dai, Yan Wang, Dongxiang Zhang, and Ee-Peng Lim. MWPToolkit: An open-source framework for deep learning-based math word problem solvers. arXiv preprint arXiv:2109.00799, 2021. URL https://arxiv.org/abs/ 2109.00799. ↗
36. Jiwei Li and Dan Jurafsky. Mutual information and diverse decoding improve neural machine translation, 2016. URL https://arxiv.org/abs/1601.00372. — [arXiv:1601.00372](https://arxiv.org/abs/1601.00372) ↗
37. Jiwei Li, Will Monroe, and Dan Jurafsky. A simple, fast diverse decoding algorithm for neural generation. CoRR, abs/1611.08562, 2016. URL http://arxiv.org/abs/1611.08562. — [arXiv:1611.08562](https://arxiv.org/abs/1611.08562) ↗
38. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017. doi: 10.18653/v1/P17-101… find
39. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. ArXiv, abs/2104.08786, 2021. find our card
40. Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. Typical decoding for natural language generation. arXiv preprint arXiv:2202.00666, 2022. — [arXiv:2202.00666](https://arxiv.org/abs/2202.00666) find
41. Shen Yun Miao, Chao Chun Liang, and Keh Yih Su. A diverse corpus for evaluating and developing English math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020. URL https://aclanthology.org/2020. acl-main.92. ↗
42. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020. find
43. Maxwell Nye, Michael Henry Tessler, Joshua B. Tenenbaum, and Brenden M. Lake. Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Sy… find
44. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080–2094, Online, June 2021. Assoc… find our card
45. Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Yan Gao, Qiang Fu, Jian-Guang Lou, and Weizhu Chen. Reasoning like program executors, 2022. find
46. Piotr Pi˛ekos, Mateusz Malinowski, and Henryk Michalewski. Measuring and improving BERT’s mathematical abilities by predicting the order of reasoning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan… find
47. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. find
48. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021. find
49. Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, and Zhiyuan Liu. NumNet: Machine reading comprehension with numerical reasoning. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP),… find
50. Subhro Roy and Dan Roth. Solving general arithmetic word problems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015. doi: 10.18653/v1/ D15-1202. URL https://aclanthology.org/D15-1202. ↗ our card
51. Jianhao Shen, Yichun Yin, Lin Li, Lifeng Shang, Xin Jiang, Ming Zhang, and Qun Liu. Generate & rank: A multi-task framework for math word problems. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 2269–2279, Punta Cana, Dominican Republic, November 2021. Association for… find
52. Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I. Wang. Natural language to code translation with execution. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 3533–3546, Abu Dhabi, United Arab Emirates, December 2022. Associatio… find
53. Keith E Stanovich and Richard F West. Individual differences in reasoning: Implications for the rationality debate? Behavioral and brain sciences, 23(5):645–665, 2000. URL https: //pubmed.ncbi.nlm.nih.gov/11301544/. find
54. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume… find
55. Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler. Unifying language learning paradigms, 2022. URL https://arxiv.org/abs/2205.05131. ↗ our card
56. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022. URL https://arxiv.org/abs/ 2201.08239. ↗ our card
57. Ashwin Vijayakumar, Michael Cogswell, Ramprasaath Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. Diverse beam search for improved description of complex scenes. Proceedings of the AAAI Conference on Artificial Intelligence, 32, Apr. 2018. URL https: //ojs.aaai.org/index.php/AAAI/a… find
58. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. Conference on Neural Information Processing Systems (NeurIPS), 2022. URL https://arxiv. org/pdf/2201.11903. find our card
59. Sean Welleck, Ilia Kulikov, Jaedeok Kim, Richard Yuanzhe Pang, and Kyunghyun Cho. Consistency of a recurrent language model with respect to incomplete decoding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5553–5568, Online, November 2020. As… find
60. Weiwen Xu, Yang Deng, Huihui Zhang, Deng Cai, and Wai Lam. Exploiting reasoning chains for multi-hop science question answering. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 1143–1156, Punta Cana, Dominican Republic, November 2021a. Association for Computational Ling… find
61. Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, and Xuedong Huang. Human parity on commonsenseqa: Augmenting self-attention with external attention, 2021b. URL https://arxiv.org/abs/ 2112.03254. ↗
62. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2369–2380, B… find our card
63. Xi Ye and Greg Durrett. The unreliability of explanations in few-shot prompting for textual reasoning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum? id=Bct2f8fRd8S. ↗ our card
64. Wenhao Yu, Chenguang Zhu, Lianhui Qin, Zhihan Zhang, Tong Zhao, and Meng Jiang. Diversifying content generation for commonsense reasoning with mixture of knowledge graph experts. In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), 2022. find
65. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Researc… find our card