基于检索增强思维提示的汉语框架语义解析方法

Summary

The paper addresses Chinese Frame Semantic Parsing (CFSP): choosing the semantic frame evoked by a target word, locating its arguments, and labeling their roles. Existing LLM prompting approaches use shallow reasoning chains and make little use of retrieval, so they perform poorly. The authors, from the University of International Relations in Beijing, propose a modular, fine-tuning-free prompting pipeline that combines retrieval-augmented generation (RAG) with chain-of-thought (CoT) reasoning. Each stage feeds the next.

• Frame identification: prompts add plain-language explanations of key concepts and a guided reasoning path. Retrieved similar examples serve as few-shots, and the frames found in those examples form a restricted candidate set.
• Argument identification: recast as generation that wraps arguments in [start]/[end] markers.
• Role identification: recast as N-way classification over the roles of the chosen frame.

On CFN 2.1, the method outperforms a ChatGPT+CoT baseline and the PKU sparse-retrieval system across three LLMs. This shows that careful prompt design can substantially narrow the gap with fine-tuned systems.

Key points

• Evaluation uses CFN 2.1, with 80% held out as a retrieval knowledge base and 20% used for testing. The tested models are GPT-3.5-turbo, Qwen1.5-32B and DeepSeek-R1-distill-Qwen-32B.
• On Qwen1.5-32B, the method beats PKU by 13.52 points in frame accuracy (67.47%), 2.24 in argument F1 (53.26%) and 5.09 in role F1 (18.62%).
• Dense retrieval works better than sparse retrieval. BGE-M3 with 10 shots, CoT and candidate frames reaches 67.43% frame accuracy, compared with 63.95% for BM25. Going to 20 shots adds only about 0.04.
• In ablations, role labeling depends heavily on frame results. Passing frame results into argument identification, however, lowered argument F1 from 53.26 to 46.30 because errors accumulated.
• Role identification remains weak overall: the best F1 is 26.46%, achieved by DeepSeek-R1-distill-Qwen-32B.