Elamur Research⌕
☰
home / papers / 2025 / 基于大语言模型的中文实体链接实证研究

基于大语言模型的中文实体链接实证研究

Summary

The study asks whether the emergent abilities of large language models (step-by-step reasoning, instruction following, in-context learning) can improve Chinese entity linking, where encoder models such as BERT tend to overfit and handle rare entities poorly. The authors build a two-stage retrieve-and-rerank pipeline. A dual encoder proposes candidate entities through maximum inner product search, and an LLM then disambiguates among them. Four reranking strategies are compared:

  • chain-of-thought knowledge augmentation of a cross-attention model (CoT-KA),
  • LoRA adapter tuning,
  • P-tuning,
  • retrieval-based ICL.

All are tested with Qwen and ChatGLM3 on the Hansel and CLEEK benchmarks. Supervised variants built on 6–7B-parameter LLMs beat the small-model baselines on every test set. Without any training, Qwen-72B nearly matches a fine-tuned Qwen-7B. The gains are largest for long-tail entities and grow with model size. An error analysis shows that most remaining mistakes concern entity granularity and entity type. The work is a practical reference for anyone linking text to knowledge bases with LLMs.

Key points

  • Supervised Qwen-7B and ChatGLM3-6B methods raise accuracy over PLM-based baselines by:
    • 3.9–11.8% on Hansel-FS,
    • 0.7–4.1% on Hansel-ZS,
    • 0.6–3.7% on CLEEK.
  • Unsupervised ICL plus chain-of-thought on Qwen-72B lands within −2.4% to +1.4% of LoRA-tuned Qwen-7B.
  • On long-tail entities, Qwen leads BERT by 11.8%, and the margin widens to 13.2% with more parameters. Parameter-efficient tuning overfits less than either BERT fine-tuning or full fine-tuning.
  • CoT-KA encodes LLM-generated reasoning chains with a separate encoder and fuses them with cross-attention features via an MLP. It is trained with a balanced scheme of hard and random negative samples.
  • Granularity errors make up 36% of failures and type errors 25%. In 56% of the granularity errors, a specific entity was linked to a more generic one.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.16383/j.aas.c240069

@inproceedings{2025,
  title = {{基于大语言模型的中文实体链接实证研究}},
  author = {徐正斐 and 辛欣},
  booktitle = {自动化学报 (Acta Automatica Sinica)},
  year = {2025},
  doi = {10.16383/j.aas.c240069},
  url = {https://doi.org/10.16383/j.aas.c240069}
}

References

48 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Guo Hao, Li Xin-Yi, Tang Jiu-Yang, Guo Yan-Ming, Zhao Xiang. Adaptive feature fusion for multi-modal entity alignment. Acta Automatica Sinica, 2024, 50(4): 758−770 (郭浩, 李欣奕, 唐九阳, 郭延明, 赵翔. 自适应特征融合的多模态实 体对齐研究. 自动化学报, 2024, 50(4): 758−770) find
  2. Hasibi F, Balog K, Bratsberg S E. Exploiting entity linking in queries for entity retrieval. In: Proceedings of the ACM International Conference on the Theory of Information Retrieval. Newark, Delaware, USA: Association for Computing Machinery, 2016. 209−218 find
  3. Liu Qiong-Xin, Wang Ya-Nan, Long Hang, Wang Jia-Sheng, Lu Shi-Shuai. Generative knowledge question answering technology based on global coverage mechanism and representation learning. Acta Automatica Sinica, 2022, 48(10): 2392−2405 (刘琼昕, 王亚男, 龙航, 王佳升, 卢士帅. 基于全局覆盖机制与表示 学习的生成式知识问答技术. 自动化学报, 2022, 48(1… find
  4. Bai J Z, Bai S, Chu Y F, Cui Z Y, Dang K, Deng X D, et al. Qwen technical report. arXiv preprint arXiv: 2309.16609, 2023. Zeng A H, Xu B, Wang B W, Zhang C H, Yin D, Zhang D, et al. ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools. arXiv preprint arXiv: 2406.12793, 2024. find our card
  5. Brown T B, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al. Language models are few-shot learners. arXiv preprint arXiv: 2005.14165, 2020. find
  6. Rawte V, Chakraborty S, Pathak A, Sarkar A, Tonmoy S M T I, Chadha A, et al. The troubling emergence of hallucination in large language models —— An extensive definition, quantification, and prescriptive remediations. In: Proceedings of the Conference on Empirical Methods in Natural Language Process… find
  7. Roberts A, Raffel C, Shazeer N. How much knowledge can you pack into the parameters of a language model? In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Virtual Event: Association for Computational Linguistics, 2020. 5418−5426 find
  8. Kandpal N, Deng H K, Roberts A, Wallace E, Raffel C. Large language models struggle to learn long-tail knowledge. arXiv preprint arXiv: 2211.08411, 2023. find our card
  9. Wang X T, Yang Q W, Qiu Y T, Liang J Q, He Q Y, Gu Z H, et al. KnowledGPT: Enhancing large language models with retrieval and storage access on knowledge bases. arXiv preprint arXiv: 2308.11761, 2023. — [arXiv:2308.11761](https://arxiv.org/abs/2308.11761) find
  10. Ganea O-E, Hofmann T. Deep joint entity disambiguation with local neural attention. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics, 2017. 2619−2629 find
  11. Le P, Titov I. Improving entity linking by modeling latent relations between mentions. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia: Association for Computational Linguistics, 2018. 1595−1604 find
  12. Jiang Sheng-Chen, Wang Hong-Bin, Yu Zheng-Tao, Xian Yan-Tuan, Wang Hong-Tao. Domain integrated-entity links based on relationship indices and representation learning. Acta Automatica Sinica, 2021, 47(10): 2376−2385 (蒋胜臣, 王红斌, 余正涛, 线岩团, 王红涛. 基于关系指数和表示学 习的领域集成实体链接. 自动化学报, 2021, 47(10): 2376−2385) find
  13. Devlin J, Chang M W, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).… find
  14. Yamada I, Washio K, Shindo H, Matsumoto Y. Global entity disambiguation with BERT. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle, USA: Association for Computational Linguistics, 2022. 3264−3271 find
  15. Wu L, Petroni F, Josifoski M, Riedel S, Zettlemoyer L. Scalable zero-shot entity linking with dense entity retrieval. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Virtual Event: Association for Computational Linguistics, 2020. 6397−6407 find our card
  16. Wei J, Wang X Z, Schuurmans D, Bosma M, Ichter B, Xia F, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 2022, 35: 24824−24837 find our card
  17. Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv: 2203.02155, 2022. find
  18. Talmor A, Tafjord O, Clark P, Goldberg Y, Berant J. Leap-of-thought: Teaching pre-trained models to systematically reason over implicit knowledge. Advances in Neural Information Processing Systems, 2020, 33: 20227−20237 find
  19. Xu Z R, Shan Z F, Li Y X, Hu B T, Qin B. Hansel: A Chinese few-shot and zero-shot entity linking benchmark. In: Proceedings of the 16th ACM International Conference on Web Search and Data Mining. Singapore: Association for Computing Machinery, 2023. 832−840 find
  20. Zeng W X, Zhao X, Tang J Y, Tan Z, Huang X Q. CLEEK: A Chinese long-text corpus for entity linking. In: Proceedings of the 12th Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association, 2020. 2026−2035 find
  21. Wu D J, Zhang J, Huang X M. Chain of thought prompting elicits knowledge augmentation. arXiv preprint arXiv: 2307.1640, 2023. find
  22. Humeau S, Shuster K, Lachaux M A, Weston J. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. arXiv preprint arXiv: 1905.01969, 2020. — [arXiv:1905.01969](https://arxiv.org/abs/1905.01969) find
  23. Houlsby N, Giurgiu A, Jastrzebski S, Morrone B, Laroussilhe Q D, Gesmundo A, et al. Parameter-efficient transfer learning for NLP. arXiv preprint arXiv: 1902.00751, 2019. find
  24. Hu E J, Shen Y L, Wallis P, Allen-Zhu Z Y, Li Y Z, Wang S, et al. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv: 2106.09685, 2022. find
  25. Liu X, Zheng Y N, Du Z X, Ding M, Qian Y J, Yang Z L, et al. GPT understands, too. AI Open, 2024, 5: 208−215 find
  26. Liu J C, Shen D H, Zhang Y Z, Dolan B, Carin L, Chen W Z. What makes good in-context examples for GPT-3? In: Proceedings of the Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures. Dublin, Ireland: Association for Computati… find our card
  27. Reimers N, Gurevych I. Sentence-BERT: Sentence embeddings using siamese BERT-networks. arXiv preprint arXiv: 1908.10084, 2019. find
  28. Huang S J, Wang B B, Qin L B, Zhao Q, Xu R F. Improving few-shot and zero-shot entity linking with coarse-to-fine lexicon-based retriever. In: Processings of the Natural Language Processing and Chinese Computing: 12th National CCF Conference. Foshan, China: 2023. 245−256 find
  29. Zhou H Y, Sun C J, Lin L, Shan L L. ERNIE-AT-CEL: A Chinese few-shot emerging entity linking model based on ERNIE and adversarial training. In: Processings of the Natural Language Processing and Chinese Computing: 12th National CCF Conference. Foshan, China: 2023. 48−56 find
  30. Xu Z, Shan Z, Hu B, Zhang M. Overview of the NLPCC 2023 shared task 6: Chinese few-shot and zero-shot entity linking. In: Proceedings of Natural Language Processing and Chinese Computing: 12th National CCF Conference. find
  31. Foshan, China: 2023. 257−265 de Cao N, Wu L, Popat K, Artetxe M, Goyal N, Plekhanov M, et al. Multilingual autoregressive entity linking. Transactions of the Association for Computational Linguistics, 2022, 10: 274−290 find
  32. Zhang Q R, Chen M S, Bukharin A, He P C, Cheng Y, Chen W Z, et al. Adaptive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv: 2303.10512v1, 2023. find our card
  33. Wei J, Tay Y, Bommasani R, Raffel C, Zoph B, Borgeaud S, et al. Emergent abilities of large language models [Online], available: https://openreview.net/forum?id=yzkSU5zdwD, January 15, 2025 ↗
  34. Bonifacio L, Abonizio H, Fadaee M, Nogueira R. InPars: Unsupervised dataset generation for information retrieval. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Madrid, Spain: Association for Computing Machinery, 2022. 2387−2392 find
  35. Ferraretto F, Laitz T, Lotufo R, Nogueira R. ExaRanker: Synthetic explanations improve neural rankers. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Taipei, China: Association for Computing Machinery, 2023. 2409−2414 find
  36. Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLoRA: Efficient finetuning of quantized LLMs. arXiv preprint arXiv: 2305.14314, 2023. find
  37. Li X L, Liang P. Prefix-tuning: Optimizing continuous prompts for generation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Virtual Event: Association for Computational Linguisti… find
  38. Lester B, Al-Rfou R, Constant N. The power of scale for parameter-efficient prompt tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Virtual Event: Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021. 3045−3059 find
  39. Liu X, Ji K X, Fu Y C, Tam W L, Du Z X, Yang Z L, et al. P-Tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Dublin, Ireland: Association for Computatio… find our card
  40. Lu Y, Bartolo M, Moore A, Riedel S, Stenetorp P. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Dublin, Ireland: Association for Computational Linguistics… find our card
  41. Zhao T, Wallace E, Feng S, Klein D, Singh S. Calibrate before use: Improving few-shot performance of language models. arXiv preprint arXiv: 2102.09690, 2021. find our card
  42. Rubin O, Herzig J, Berant J. Learning to retrieve prompts for in-context learning. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle, USA: Association for Computational Linguistics, 2022. 2655−2671 find our card
  43. Min S, Lewis M, Zettlemoyer L, Hajishirzi H. MetaICL: Learning to learn in context. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle, USA: Association for Computational Linguistics, 2022. 2791−2809 find
  44. Cho Y M, Zhang L, Callison-Burch C. Unsupervised entity linking with guided summarization and multiple-choice selection. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2022. 9394−9401 find
  45. Shi S B, Xu Z R, Hu B T, Zhang M. Generative multimodal entity linking. In: Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). find
  46. Torino, Italy: ELRA and ICCL, 2024. 7654−7665 de Cao N, Izacard G, Riedel S, Petroni F. Autoregressive entity retrieval. In: Proceedings of the 9th International Conference on Learning Representations. find
  47. Virtual Event: ICLR, 2021. find
  48. 徐正斐 北京理工大学计算机学院硕 士研究生. 主要研究方向为知识工程. E-mail: zhengfei@bit.edu.cn (XU Zheng-Fei Master student at the School of Computer Science and Technology, Beijing Institute of Technology. His main research interest is knowledge engineering.) 辛 欣 北京理工大学计算机学院副 教授. 主要研究方向为自然语言处理, 知识工程, 信息检索. 本文通信作者. E-mail: xxin@b… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/xu-2025-empirical-study-chinese/