Elamur Research⌕
☰
home / papers / 2024 / 基于大语言模型的事件常识知识图谱扩展方法

基于大语言模型的事件常识知识图谱扩展方法

Summary

Machines struggle to acquire commonsense knowledge about events, and large event-centric commonsense graphs are rare. Existing resources such as ATOMIC and ConceptNet cover only a few event relation types. The authors start from their manually curated Chinese seed ECKG (26,606 triples) and use GPT-series LLMs to expand four relations: causal, temporal, conditional and sub-event.

The pipeline has three stages. First, relation-specific ek-prompts ask GPT-4-Turbo for head and tail events of each seed event; each prompt contains a definition, an optional example, a question and output constraints. Second, the correct generated triples are merged with seed triples to fine-tune GPT-3.5-Turbo into a generator (ft-gen) and a correctness checker (ft-ver). Third, an event-sharing mechanism measures similarity between seed events and lets near-duplicate events exchange their linked events under the same relation, so that similar events receive consistent triples.

The expansion adds about 2.4 million high-quality triples. This gives Chinese NLP a far larger event commonsense resource and a reusable LLM-based recipe for growing knowledge graphs.

Key points

  • Seed ECKG has 26,606 hand-built Chinese event triples. The expansion contributes 2,433,012 new triples.
  • Temporal triples reach the highest accuracy, 98.2%.
  • Before sub-event expansion, an SVM classifier filters out events unlikely to have sub-events. It is trained on data labeled using 11 hand-defined feature types, such as states, emotions, perception and possession.
  • Event similarity combines text-embedding-ada-002 embeddings with semantic role labeling, semantic roles including sentence constituents, SRL structure similarity and antonym features.
  • Published in 软件学报 / Journal of Software 2025, 36(9): 4153–4186 (online 31 Dec 2024), by Huang Qiao-Juan, Cao Cun-Gen, Wang Ya and Wang Shi (ICT CAS).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.13328/j.cnki.jos.007262

@article{2024,
  title = {{基于大语言模型的事件常识知识图谱扩展方法}},
  author = {黄俏娟 and 曹存根 and 王亚 and 王石},
  journal = {软件学报 (Journal of Software)},
  year = {2024},
  doi = {10.13328/j.cnki.jos.007262},
  url = {https://doi.org/10.13328/j.cnki.jos.007262}
}

References

72 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nguyen TP, Razniewski S, Weikum G. Advanced semantics for commonsense knowledge extraction. In: Proc. of the 30th Web Conf. 结果: Y: 康复(张姨) 愿望知识库 事实: 家人(张姨, 李红). 规则: 愿望(X, 康复(Y)) :- 是一个(X, 人), 是一个(Y, 人), 家人(X, Y) Y: (人) 查询: 愿望(李红, Y). 多知识库推理 Ljubljana: ACM, 2021. 2636–2647. [doi: 10.1145/3442381.34498… find
  2. Zellers R, Bisk Y, Farhadi A, Choi Y. From recognition to cognition: Visual commonsense reasoning. In: Proc. of the 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019. 6713–6724. [doi: 10.1109/CVPR.2019.00688] find
  3. Vijayaraghavan P, Roy D. Modeling human motives and emotions from personal narratives using external knowledge and entity tracking. In: Proc. of the 30th Web Conf. Ljubljana: ACM, 2021. 529–540. [doi: 10.1145/3442381.3449997] — [doi:10.1145/3442381.3449997](https://doi.org/10.1145/3442381.3449997) find
  4. Suchanek FM, Kasneci G, Weikum G. YAGO: A core of semantic knowledge. In: Proc. of the 16th Int’l Conf. on World Wide Web. 2007. 697–706. [doi: 10.1145/1242572.1242667] find
  5. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z. DBpedia: A nucleus for a Web of open data. In: Proc. of the 6th Int’l Semantic Web Conf., the 2nd Asian Semantic Web Conf. Busan: Springer, 2007. 722–735. [doi: 10.1007/978-3-540-76298-0_52] — [doi:10.1007/978-3-540-76298-0_52](https://doi.org/10.1007/978-3-540-76298-0_52) find
  6. Vrandečić D, Krötzsch M. Wikidata: A free collaborative knowledgebase. Communications of the ACM, 2014, 57(10): 78–85. [doi: 10.1145/2629489] find
  7. Glavaš G, Šnajder J. Event graphs for information retrieval and multi-document summarization. Expert Systems with Applications, 2014, 41(15): 6904–6916. [doi: 10.1016/j.eswa.2014.04.004] — [doi:10.1016/j.eswa.2014.04.004](https://doi.org/10.1016/j.eswa.2014.04.004) find
  8. Zhang HM, Liu X, Pan HJ, Song YQ, Leung CWK. ASER: A large-scale eventuality knowledge graph. In: Proc. of the 29th Web Conf. Taipei: ACM, 2020. 201–211. [doi: 10.1145/3366423.3380107] find
  9. Ding X, Li Z, Liu T, Liao K. ELG: An event logic graph. arXiv:1907.08015, 2019. — [arXiv:1907.08015](https://arxiv.org/abs/1907.08015) find
  10. Sap M, Le Bras R, Allaway E, Bhagavatula C, Lourie N, Rashkin H, Roof B, Smith NA, Choi Y. ATOMIC: An atlas of machine commonsense for if-then reasoning. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. Honolulu: AAAI, 2019. 3027–3035. [doi: 10.1609/aaai.v33i01.33013027] find
  11. Liu H, Singh P. ConceptNet—A practical commonsense reasoning tool-kit. BT Technology Journal, 2004, 22(4): 211–226. [doi: 10.1023/B:BTTJ.0000047600.45421.6d] — [doi:10.1023/b:bttj.0000047600.45421.6d](https://doi.org/10.1023/b:bttj.0000047600.45421.6d) find
  12. Wang Y, Cao CG, Cao YT, Wang S. A property-based method for acquiring commonsense knowledge. In: Proc. of the 14th Int’l Conf. on KSEM Science, Engineering and Management. Tokyo: Springer, 2021. 53–65. [doi: 10.1007/978-3-030-82136-4_5] — [doi:10.1007/978-3-030-82136-4_5](https://doi.org/10.1007/978-3-030-82136-4_5) find
  13. Wang Y, Cao CG, Chen ZW, Wang S. ECCKG: An eventuality-centric commonsense knowledge graph. In: Proc. of the 15th Int’l Conf. on Knowledge Science, Engineering and Management. Singapore: Springer, 2022. 568–584. [doi: 10.1007/978-3-031-10983-6_44] — [doi:10.1007/978-3-031-10983-6_44](https://doi.org/10.1007/978-3-031-10983-6_44) find
  14. Song H. Research on Chinese semantic parsing and knowledge acquisition method based on semantic roles [Ph.D. Thesis]. Beijing: University of Chinese Academy of Sciences, 2022 (in Chinese with English abstract). find
  15. Wang Y, Cao CG, Wang S. Commonsense knowledge acquisition via semantic roles in events. Journal of Chinese Information Processing, 2023, 37(6): 77–88 (in Chinese with English abstract). [doi: 10.3969/j.issn.1003-0077.2023.06.008] — [doi:10.3969/j.issn.1003-0077.2023.06.008](https://doi.org/10.3969/j.issn.1003-0077.2023.06.008) find
  16. Lenat DB. CYC: A large-scale investment in knowledge infrastructure. Communications of the ACM, 1995, 38(11): 33–38. [doi: 10.1145/219717.219745] find
  17. Fellbaum C, Miller G. WordNet: An Electronic Lexical Database. Cambridge: MIT Press, 1998. find
  18. Singh P, Lin T, Mueller ET, Lim G, Perkins T, Zhu WL. Open mind common sense: Knowledge acquisition from the general public. In: Meersman R, Tari Z, eds. On the Move to Meaningful Internet Systems 2002: CoopIS, DOA, and ODBASE: Confederated Int’l Conf. CoopIS, DOA, and ODBASE 2002 Proc. Berlin: Spri… find
  19. Zhang HM, Khashabi D, Song YQ, Roth D. TransOMCS: From linguistic graphs to commonsense knowledge. In: Proc. of the 29th Int’l Joint Conf. on Artificial Intelligence. Yokohama, 2021. 554. find
  20. Romero J, Razniewski S, Pal K, Pan JZ, Sakhadeo A, Weikum G. Commonsense properties from query logs and question answering forums. In: Proc. of the 28th ACM Int’l Conf. on Information and Knowledge Management. Beijing: ACM, 2019. 1411–1420. [doi: 10.1145/3357384.3357955] find
  21. Cao YN. Research on causal knowledge acquisition [Ph.D. Thesis]. Beijing: University of Chinese Academy of Sciences, 2012 (in Chinese with English abstract). find
  22. Bosselut A, Rashkin H, Sap M, Malaviya C, Celikyilmaz A, Choi Y. COMET: Commonsense Transformers for automatic knowledge graph construction. In: Proc. of the 57th Annual Meeting of the Association for Computational Linguistics. Florence: ACL, 2019. 4762–4779. find our card
  23. Tandon N, De Melo G, Suchanek F, Weikum G. WebChild: Harvesting and organizing commonsense knowledge from the Web. In: Proc. of the 7th ACM Int’l Conf. on Web Search and Data Mining. New York: ACM, 2014. 523–532. [doi: 10.1145/2556195.2556245] find
  24. Chklovski T. Learner: A system for acquiring commonsense knowledge by analogy. In: Proc. of the 2nd Int’l Conf. on Knowledge Capture. Sanibel Island: ACM, 2003. 4–12. [doi: 10.1145/945645.945650] — [doi:10.1145/945645.945650](https://doi.org/10.1145/945645.945650) find
  25. Cao C, Cao CG, Zang LJ, Wang S. An interactive method for acquiring event-based commonsense knowledge. Journal of Chinese Information Processing, 2016, 30(3): 125–132 (in Chinese with English abstract). find
  26. Wang Y. Research on commonsense knowledge acquisition [Ph.D. Thesis]. Beijing: University of Chinese Academy of Sciences, 2022 (in Chinese with English abstract). find
  27. Deng P. A study of event graph completion and reasoning based on semantic enhancement [MS. Thesis]. Beijing: Institute of Computer Technology, Chinese Academy of Sciences, 2023 (in Chinese with English abstract). find
  28. Meilicke C, Fink M, Wang YJ, Ruffinelli D, Gemulla R, Stuckenschmidt H. Fine-grained evaluation of rule- and embedding-based systems for knowledge graph completion. In: Proc. of the 17th Int’l Semantic Web Conf. Monterey: Springer, 2018. 3–20. [doi: 10.1007/978-3-030-00671-6_1] find
  29. Li X, Taheri A, Tu LF, Gimpel K. Commonsense knowledge base completion. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin: ACM, 2016. 1445–1455. [doi: 10.18653/v1/P16-1137] find
  30. Malaviya C, Bhagavatula C, Bosselut A, Choi Y. Commonsense knowledge base completion with structural and semantic context. In: Proc. of the 34th AAAI Conf. on Artificial Intelligence. New York: AAAI, 2020. 2925–2933. [doi: 10.1609/aaai.v34i03.5684] — [doi:10.1609/aaai.v34i03.5684](https://doi.org/10.1609/aaai.v34i03.5684) find
  31. Niu GL, Li B. Logic and commonsense-guided temporal knowledge graph completion. In: Proc. of the 37th AAAI Conf. on Artificial Intelligence. Washington: AAAI, 2023. 4569–4577. [doi: 10.1609/aaai.v37i4.25579] — [doi:10.1609/aaai.v37i4.25579](https://doi.org/10.1609/aaai.v37i4.25579) find
  32. Wang B, Wang GT, Huang J, You JX, Leskovec J, Jay Kuo CC. Inductive learning on commonsense knowledge graph completion. In: Proc. of the 2021 Int’l Joint Conf. on Neural Networks. Shenzhen: IEEE, 2021. 1–8. [doi: 10.1109/IJCNN52387.2021.9534355] — [doi:10.1109/ijcnn52387.2021.9534355](https://doi.org/10.1109/ijcnn52387.2021.9534355) find
  33. Bosselut A, Le Bras R, Choi Y. Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering. In: Proc. of the 35th AAAI Conf. on Artificial Intelligence. AAAI, 2021. 4923–4931. [doi: 10.1609/aaai.v35i6.16625] find
  34. Zhang NY, Xie X, Chen X, Deng SM, Ye HB, Chen HJ. Knowledge collaborative fine-tuning for low-resource knowledge graph completion. Ruan Jian Xue Bao/Journal of Software, 2022, 33(10): 3531–3545 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6628.htm [doi: 10.13328/j.cnki.jos.006… — [doi:10.13328/j.cnki.jos.006628](https://doi.org/10.13328/j.cnki.jos.006628) ↗
  35. Zhang YC, Chen Z, Guo LB, Xu YJ, Zhang W, Chen HJ. Making large language models perform better in knowledge graph completion. arXiv:2310.06671, 2023. — [arXiv:2310.06671](https://arxiv.org/abs/2310.06671) find
  36. Wei YB, Huang QS, Zhang Y, Kwok JT. KICGPT: Large language model with knowledge in context for knowledge graph completion. In: Proc. of the 2024 Findings of the Association for Computational Linguistics. Singapore: ACL, 2024. 8667–8683. [doi: 10.18653/v1/2023.findings-emnlp.580] find
  37. Luo RL, Gu TL, Li HL, Li JZ, Lin ZC, Li JY, Yang YJ. Chain of history: Learning and forecasting with LLMs for temporal knowledge graph completion. arXiv:2401.06072, 2024. find
  38. Yao L, Peng JZ, Mao CS, Luo Y. Exploring large language models for knowledge graph completion. arXiv:2308.13916, 2023. — [arXiv:2308.13916](https://arxiv.org/abs/2308.13916) find
  39. Pan SR, Luo LH, Wang YF, Chen C, Wang JP, Wu XD. Unifying large language models and knowledge graphs: A roadmap. IEEE Trans. on Knowledge and Data Engineering, 2024, 36(7): 3580–3599. [doi: 10.1109/TKDE.2024.3352100] find our card
  40. Li DW, Tan Z, Chen TL, Liu H. Contextualization distillation from large language model for knowledge graph completion. In: Proc. of the 2024 Findings of the Association for Computational Linguistics. St. Julian’s: ACL, 2024. 458–477. find
  41. OpenAI. GPT-4 technical report. arXiv:2303.08774, 2023. find our card
  42. Liu YH, Ott M, Goyal N, Du JF, Joshi M, Chen DQ, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692, 2019. find
  43. Galárraga LA, Teflioudi C, Hose K, Suchanek F. AMIE: Association rule mining under incomplete evidence in ontological knowledge bases. In: Proc. of the 22nd Int’l Conf. on World Wide Web. Rio de Janeiro: ACM, 2013. 413–422. [doi: 10.1145/2488388.2488425] find
  44. Wang Y, Cao CG. Research on categorization of events based on event attributes. Journal of Chinese Information Processing, 2020, 34(10): 39–50 (in Chinese with English abstract). [doi: 10.3969/j.issn.1003-0077.2020.10.006] — [doi:10.3969/j.issn.1003-0077.2020.10.006](https://doi.org/10.3969/j.issn.1003-0077.2020.10.006) find
  45. Song H, Cao CG, Wang Y, Wang S. A fine-grained annotated dataset for Chinese semantic-role labeling. Journal of Chinese Information Processing, 2023, 37(1): 16–32 (in Chinese with English abstract). [doi: 10.3969/j.issn.1003-0077.2023.01.002] — [doi:10.3969/j.issn.1003-0077.2023.01.002](https://doi.org/10.3969/j.issn.1003-0077.2023.01.002) find
  46. Cortes C, Vapnik V. Support-vector networks. Machine Learning, 1995, 20(3): 273–297. [doi: 10.1007/BF00994018] find
  47. OpenAI. Introduction to GPT-4 and GPT-4-Turbo. 2024. https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo ↗
  48. Song H, Cao CG, Wang Y, Wang S. Construction of a finely-grained training dataset for Chinese semantic-role labeling. Journal of Chinese Information Processing, 2022, 36(12): 52–66, 73 (in Chinese with English abstract). [doi: 10.3969/j.issn.1003-0077.2022.12.006] — [doi:10.3969/j.issn.1003-0077.2022.12.006](https://doi.org/10.3969/j.issn.1003-0077.2022.12.006) find
  49. Lu C. Linguistics for Knowledge Engineering. Beijing: Tsinghua University Press, 2010 (in Chinese). find
  50. Wang Y. Research on common sense knowledge acquisition methods based on semantic classification [MS. Thesis]. Guilin: Guangxi Normal University, 2015 (in Chinese with English abstract). find
  51. Zhu XY. Studies on Semantic Structure Patterns of Sentences in Modern Chinese. Beijing: Peking University Press, 2001 (in Chinese). find
  52. Yuan YL. Research on Chinese Computational Linguistics Based on Cognition. Beijing: Peking University Press, 2008 (in Chinese). find
  53. Liu MF, Hu HJ. Research on Event Semantics Based on Cognition and Computation. Beijing: Science Press, 2013 (in Chinese). find
  54. Zang LJ, Wang WM, Wang Y, Fang F, Cao C, Wu XL, Zhou Q, Lu Y, Li TY, Cao CG. A Chinese framework of semantic taxonomy and description: Preliminary experimental evaluation using web information extraction. In: Proc. of the 8th Int’l Conf. on Knowledge Science, Engineering and Management. Chongqing: S… — [doi:10.1007/978-3-319-25159-2_25](https://doi.org/10.1007/978-3-319-25159-2_25) find
  55. Bai XP. Adding global semantic roles to the Chinese proposition bank to facilitate Chinese language studies. Linguistic Sciences, 2017, 16(5): 481–492 (in Chinese with English abstract). [doi: 10.7509/j.linsci.201705.031178] — [doi:10.7509/j.linsci.201705.031178](https://doi.org/10.7509/j.linsci.201705.031178) find
  56. Wang CW, Qian QQ, Xun ED, Xing D, Li M, Rao GQ. Construction of semantic role bank for Chinese verbs from the perspective of ternary collocation. Journal of Chinese Information Processing, 2020, 34(9): 19–27 (in Chinese with English abstract). [doi: 10.3969/j. issn.1003-0077.2020.09.004] — [doi:10.3969/j](https://doi.org/10.3969/j) find
  57. Wang LJ. Research on Chinese semantic dependency analysis [MS. Thesis]. Harbin: Harbin Institute of Technology, 2010 (in Chinese with English abstract). find
  58. SUMO. 2023. http://www.ontologyportal.org ↗
  59. Song H, Wang S, Liu Y, Wang Y. Predicate-attention neural model for Chinese semantic role labeling. Computers and Electrical Engineering, 2022, 99: 107741. [doi: 10.1016/j.compeleceng.2022.107741] — [doi:10.1016/j.compeleceng.2022.107741](https://doi.org/10.1016/j.compeleceng.2022.107741) find
  60. Chen Z W. A Study of event graph construction method based on semantic parsing [MS. Thesis]. Beijing: Institute of Computer Technology, Chinese Academy of Sciences, 2023 (in Chinese with English abstract). find
  61. Zhang TY, Kishore V, Wu F, Weinberger KQ, Artzi Y. BERTScore: Evaluating text generation with BERT. In: Proc. of the 8th Int’l Conf. on Learning Representations. Addis Ababa: OpenReview.net, 2020. find
  62. Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional Transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1 (Long and Short Papers). Minneapol… find
  63. Alpaydın E. Introduction to Machine Learning. 4th ed., Cambridge: MIT Press, 2020. 2–12. find
  64. Wielemaker J, Schrijvers T, Triska M, Lager T. SWI-Prolog. Theory and Practice of Logic Programming, 2012, 12(1–2): 67–96. [doi: 10.1017/S1471068411000494] — [doi:10.1017/s1471068411000494](https://doi.org/10.1017/s1471068411000494) find
  65. Anthropic. 2024. https://www.anthropic.com/claude ↗
  66. iFLYTEK. 2024. https://www.xfyun.cn/doc/spark/Web.html ↗
  67. Alibaba Cloud. 2024. https://tongyi.aliyun.com/qianwen ↗
  68. Baidu. 2024. https://yiyan.baidu.com/ ↗
  69. Banarescu L, Bonial C, Cai S, Georgescu M, Griffitt K, Hermjakob U, Knight K, Koehn P, Palmer M, Schneider N. Abstract meaning representation for sembanking. In: Proc. of the 7th Linguistic Annotation Workshop and Interoperability with Discourse. Sofia: ACL, 2013. 178–186. find our card
  70. He H, Choi JD. The stem cell hypothesis: Dilemma behind multi-task learning with Transformer encoders. arXiv:2109.06939, 2021. find
  71. Nivre J. Dependency grammar and dependency parsing. MSI Report, 2005, 5133(1959): 1–32. find
  72. Chaudhari S, Aggarwal P, Murahari V, Rajpurohit T, Kalyan A, Narasimhan K, Deshpande A, da Silva BC. RLHF Deciphered: A critical analysis of reinforcement learning from human feedback for LLMs. arXiv:2404.08555, 2024. 附中文参考文献: [14] 宋衡. 基于语义角色的中文语义解析与知识获取方法研究 [博士学位论文]. 北京: 中国科学院大学, 2022. [15] 王亚, 曹存根… — [doi:10.3969/j.issn.1003-0077.2023.06.008](https://doi.org/10.3969/j.issn.1003-0077.2023.06.008) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/huang-2024-method-expanding-event/