面向小规模大语言模型推理优化的推理路径排序方法

Summary

Small-scale LLMs (SLLMs) are much cheaper to deploy than large LLMs but reason noticeably worse. Prompting methods such as manual CoT help, but writing good demonstrations takes human effort and domain expertise. The authors propose a two-stage pipeline that needs no human involvement. First, one SLLM samples several candidate reasoning paths using a zero-shot step-by-step prompt. Next, a ranking module called SSVRP (Step-Semantic Verifier on Reasoning Paths) picks the best path. Finally, a second SLLM produces the answer from that path. SSVRP splits both the question and each path into sentences. It scores a path by the mean cosine similarity between question sentences and reasoning steps, using a DeBERTa-base encoder tuned with SimCSE-style unsupervised contrastive learning. All training data for the verifier is generated by SLLMs. Across four arithmetic and commonsense benchmarks, the method reaches state-of-the-art results among SLLM-based approaches on three tasks. Its performance is close to LLM-based Auto-CoT, which suggests cheap path selection can narrow the gap between small and large models.

Key points

• Benchmarks and results: The method was tested on GSM8K, MultiArith, SVAMP and CommonsenseQA, with reported average accuracies of 54.3%, 90.6%, 64.3% and 63.7%. It sets new SOTA on 3 of the 4 tasks.
• Verifier training data: LLaMA2, Baichuan2 and ChatGLM3 each produced 3 paths per question, giving 9 paths per question. This covered 8,593 arithmetic and 9,741 commonsense questions, for 77,257 and 87,669 paths respectively.
• Baselines:
  • GPT-3 (davinci, code-davinci-002) with Greedy Decode or Self-Consistency.
  • FlanT5-XL (3B), LLaMA2 (7B), Baichuan2 (7B) and ChatGLM3 (6B) with Greedy Decode, Self-Consistency or DIVERSE.
• Ablations: SSVRP was compared with random path selection and with SVRP, a variant that scores whole paths without splitting them into steps.
• Analysis finding: SLLMs differ strongly when generating reasoning paths. Given the same path, however, they answer almost identically, so answer quality depends mainly on path quality.