Elamur Research⌕
☰
home / papers / 2023 / Large Knowledge Model: Perspectives and Challenges

Large Knowledge Model: Perspectives and Challenges

Paper · 2023 · Huajun Chen

Summary

This position paper by Huajun Chen (Zhejiang University) asks how large models can handle human knowledge, which comes in many structured forms and not only as word sequences. It contains no new dataset or benchmark. Instead, it surveys and organizes research in two directions. First, it reviews ways symbolic resources like Knowledge Graphs (KGs) can improve LLMs: adding knowledge while the model is being pre-trained, enriching prompts with knowledge, giving chain-of-thought (CoT) reasoning an explicit structure, modifying the knowledge a model stores, using tools grounded in semantics, and building agents that are aware of knowledge. Second, it covers how LLMs can help classic knowledge bases, for example by building and controlling KGs, pre-training on structured knowledge, and supporting symbolic reasoning. A central argument is that expressiveness and scale pull against each other. Rich formalisms such as ontologies give precision but are hard to scale, while plain text scales but weakens fine-grained reasoning. The author therefore proposes the Large Knowledge Model (LKM), designed natively for diverse knowledge structures, and summarizes its requirements in a five-"A" principle. The paper is useful as a roadmap for combining KG and LLM work.

Key points

  • Expressiveness vs. scale. The paper frames symbolic KGs and LLMs as complementary.
    • KGs offer precision, traceable reasoning and interpretability, but they are hard to scale.
    • LLMs offer broad coverage and generalization, but they hallucinate and struggle with deep logic.
  • Structure and reasoning. The cited "Reasoning with Program of Thoughts" study scores code prompts by structural and logical complexity. Loops score higher than plain assignments. Prompts with higher scores yielded better reasoning across benchmarks.
  • KG injection into pre-trained models. Approaches are grouped by layer:
    • input layer, e.g., K-BERT and CoLAKE, which append linked triples to the text;
    • intermediate (architecture) layer;
    • objective layer, e.g., a KGE loss combined with MLM.
  • Commonsense resources. The author proposes merging existing knowledge bases with LLMs to build a large commonsense or "world" model. Resources named include Cyc, ConceptNet, Yago, DBPedia, Wikidata, BabelNet, ATOMIC and ASTER.
  • Five-"A" principle for an LKM.
    • Augmented pretraining
    • Authentic knowledge, achieved by decoupling the knowledge base from the LM
    • Accountable reasoning
    • Abundant coverage, achieved through agent communities
    • Alignment with knowledge, including ethics knowledge bases

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Zhejiang UniversityHangzhou · China236 works · 14,072 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 worksComputer Vision and Pattern Recognition · 3,621 works

Cite

Cite as Huajun Chen. 2023. Large Knowledge Model: Perspectives and Challenges. arXiv:2312.02706.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3724/2096-7004.di.2024.0001 · arXiv 2312.02706

@misc{chen2023large,
  title = {{Large Knowledge Model: Perspectives and Challenges}},
  author = {Huajun Chen},
  year = {2023},
  doi = {10.3724/2096-7004.di.2024.0001},
  eprint = {2312.02706},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3724/2096-7004.di.2024.0001}
}

References

110 references, 39 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. G. Yule. The study of language (6th ed.). Cambridge University Press., 2019. find
  2. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. CoRR, abs/2305.20050, 2023.
  3. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In NeurIPS Datasets and Benchmarks, 2021.
  4. Stanislaw Jastrzebski, Damian Lesniak, and Wojciech Marian Czarnecki. Learning to SMILE(S). CoRR, abs/1602.06289, 2016. — [arXiv:1602.06289](https://arxiv.org/abs/1602.06289) find
  5. Zheng Xu, Sheng Wang, Feiyun Zhu, and Junzhou Huang. Seq2seq fingerprint: An unsupervised deep molecular embedding for drug discovery. In BCB, pages 285–294. ACM, 2017. find
  6. Michael Ashburner, Catherine A. Ball, Judith A. Blake, David Botstein, Heather Butler, J. Michael Cherry, Allan P. Davis, Kara Dolinski, Selina S. Dwight, Janan T. Eppig, Midori A. Harris, David P. Hill, Laurie Issel-Tarver, Andrew Kasarskis, Suzanna Lewis, John C. Matese, Joel E. Richardson, Martin… find
  7. The Gene Ontology Consortium. The gene ontology resource: 20 years and still going strong. Nucleic Acids Res., 47(Database-Issue):D330–D338, 2019. find
  8. Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Qiang Zhang, Jiazhang Lian, and Huajun Chen. Ontoprotein: Protein pretraining with gene ontology embedding. In ICLR. OpenReview.net, 2022. find
  9. Randall Davis, Howard E. Shrobe, and Peter Szolovits. What is a knowledge representation? AI Mag., 14(1):17–33, 1993. find
  10. Franz Baader, Ian Horrocks, Carsten Lutz, and Ulrike Sattler. An Introduction to Description Logic. Cambridge University Press, 2017. find
  11. Philipp Körner, Michael Leuschel, João Barbosa, Vítor Santos Costa, Verónica Dahl, Manuel V. Hermenegildo, José F. Morales, Jan Wielemaker, Daniel Diaz, and Salvador Abreu. Fifty years of prolog and beyond. Theory Pract. Log. Program., 22(6):776–858, 2022. find
  12. William A. Woods. What’s in a link: Foundations for semantic networks. pages 35–84, 1975. find
  13. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of large l… — A Survey of Large Language Models
  14. Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, and Xia Hu. Harnessing the power of llms in practice: A survey on chatgpt and beyond. CoRR, abs/2304.13712, 2023.
  15. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020.
  16. Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Heung-Yeung Shum, and Jian Guo. Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. CoRR, abs/2307.07697, 2023. find
  17. Yilin Wen, Zifeng Wang, and Jimeng Sun. Mindmap: Knowledge graph prompting sparks graph of thoughts in large language models. CoRR, abs/2308.09729, 2023.
  18. Zeyuan Wang, Qiang Zhang, Keyan Ding, Ming Qin, Xiang Zhuang, Xiaotong Li, and Huajun Chen. Instructprotein: Aligning human and protein language via knowledge instruction. CoRR, abs/2310.03269, 2023. find
  19. Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Lidong Bing, Shafiq R. Joty, and Soujanya Poria. Chain of knowledge: A framework for grounding large language models with structured knowledge bases. CoRR, abs/2305.13269, 2023.
  20. Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. Unifying large language models and knowledge graphs: A roadmap. CoRR, abs/2306.08302, 2023.
  21. Ran Xu, Hejie Cui, Yue Yu, Xuan Kan, Wenqi Shi, Yuchen Zhuang, Wei Jin, Joyce C. Ho, and Carl Yang. Knowledge-infused prompting: Assessing and advancing clinical text data generation with large language models. CoRR, abs/2311.00287, 2023. find
  22. Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. CoRR, abs/2211.12588, 2022. find
  23. Zhen Bi, Ningyu Zhang, Yinuo Jiang, Shumin Deng, Guozhou Zheng, and Huajun Chen. When do program-of-thoughts work for reasoning? CoRR, abs/2308.15452, 2023. — [arXiv:2308.15452](https://arxiv.org/abs/2308.15452) find
  24. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022.
  25. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In ICLR. OpenReview.net, 2023.
  26. Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. K-BERT: enabling language representation with knowledge graph. In AAAI, pages 2901–2908. AAAI Press, 2020.
  27. Tianxiang Sun, Yunfan Shao, Xipeng Qiu, Qipeng Guo, Yaru Hu, Xuanjing Huang, and Zheng Zhang. Colake: Contextualized language and knowledge embedding. In COLING, pages 3660–3670. International Committee on Computational Linguistics, 2020.
  28. Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. ERNIE: enhanced language representation with informative entities. In ACL (1), pages 1441–1451. Association for Computational Linguistics, 2019. find
  29. Matthew E. Peters, Mark Neumann, Robert L. Logan IV, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. Knowledge enhanced contextual word representations. In EMNLP/IJCNLP (1), pages 43–54. Association for Computational Linguistics, 2019. find
  30. Ye Liu, Yao Wan, Lifang He, Hao Peng, and Philip S. Yu. KG-BART: knowledge graph-augmented BART for generative commonsense reasoning. In AAAI, pages 6418–6425. AAAI Press, 2021. find
  31. An Yang, Quan Wang, Jing Liu, Kai Liu, Yajuan Lyu, Hua Wu, Qiaoqiao She, and Sujian Li. Enhancing pre-trained language representations with rich knowledge for machine reading comprehension. In ACL (1), pages 2346–2357. Association for Computational Linguistics, 2019. find
  32. Bin He, Di Zhou, Jinghui Xiao, Xin Jiang, Qun Liu, Nicholas Jing Yuan, and Tong Xu. Integrating graph contextualized knowledge into pre-trained language models. In EMNLP (Findings), volume EMNLP 2020 of Findings of ACL, pages 2281–2290. Association for Computational Linguistics, 2020. find
  33. Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. KEPLER: A unified model for knowledge embedding and pre-trained language representation. Trans. Assoc. Comput. Linguistics, 9:176–194, 2021. find
  34. Wenhan Xiong, Jingfei Du, William Yang Wang, and Veselin Stoyanov. Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model. In ICLR. OpenReview.net, 2020.
  35. Donghan Yu, Chenguang Zhu, Yiming Yang, and Michael Zeng. JAKET: joint pre-training of knowledge graph and language understanding. In AAAI, pages 11630–11638. AAAI Press, 2022. find
  36. Matthew B. A. McDermott, Brendan Yap, Peter Szolovits, and Marinka Zitnik. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature Machine Intelligence, 5:612–621, 2023. find
  37. Weizhe Yuan and Pengfei Liu. restructured pre-training. CoRR, abs/2206.11147, 2022. — [arXiv:2206.11147](https://arxiv.org/abs/2206.11147) find
  38. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Train… — Training language models to follow instructions with human feedback
  39. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. In ICLR. OpenReview.net, 2022.
  40. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. In ICLR. OpenReview.net, 2023.
  41. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In ICLR. OpenReview.net, 2023.
  42. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In EMNLP (1), pages 4222–4235. Association for Computational Linguistics, 2020.
  43. Benfeng Xu, Quan Wang, Zhendong Mao, Yajuan Lyu, Qiaoqiao She, and Yongdong Zhang. knn prompting: Beyond-context learning with calibration-free nearest neighbor inference. In ICLR. OpenReview.net, 2023. find
  44. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. CoRR, abs/2305.10601, 2023. find
  45. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. CoRR, abs/2308.09687, 2023. find
  46. Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. PTR: prompt tuning with rules for text classification. AI Open, 3:182–192, 2022. find
  47. Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In WWW, pages 2778–2788. ACM, 2022. find
  48. Yin Fang, Qiang Zhang, Ningyu Zhang, Zhuo Chen, Xiang Zhuang, Xin Shao, Xiaohui Fan, and Huajun Chen. Knowledge graph-enhanced molecular contrastive learning with functional prompt. Nature Machine Intelligence, 5:542–553, 2023. find
  49. Ningyu Zhang, Jintian Zhang, Xiaohan Wang, Honghao Gui, Kangwei Liu, Yinuo Jiang, Xiang Chen, Shengyu Mao, Shuofei Qiao, Yuqi Zhu, Zhen Bi, Jing Chen, Xiaozhuan Liang, Yixin Ou, Runnan Fang, Zekun Xi, Xin Xu, Lei Li, Peng Wang, Mengru Wang, Yunzhi Yao, Bozhong Tian, Yin Fang, Guozhou Zheng, and Huaj… find
  50. Zeyuan Wang, Qiang Zhang, Keyan Ding, Ming Qin, Xiang Zhuang, Xiaotong Li, and Huajun Chen. Instructprotein: Aligning human and protein language via knowledge instruction. CoRR, abs/2310.03269, 2023. find
  51. Masoomali Fatehkia, Ji Kim Lucas, and Sanjay Chawla. T-RAG: lessons from the LLM trenches. CoRR, abs/2402.07483, 2024. — [arXiv:2402.07483](https://arxiv.org/abs/2402.07483) find
  52. Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, and Michael Zeng. Kg-fid: Infusing knowledge graph in fusion-in-decoder for open-domain question answering. In ACL (1), pages 4961–4974. Association for Computational Linguistics, 2022.
  53. Xiang Chen, Lei Li, Ningyu Zhang, Xiaozhuan Liang, Shumin Deng, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. Decoupling knowledge from memorization: Retrieval-augmented prompt learning. In NeurIPS, 2022. find
  54. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. CoRR, abs/2308.05374, 2023.
  55. Garima Agrawal, Tharindu Kumarage, Zeyad Alghami, and Huan Liu. Can knowledge graphs reduce hallucinations in llms? : A survey. CoRR, abs/2311.07914, 2023. find
  56. Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, and Huajun Chen. Easyedit: An easy-to-use knowledge editing framework for large language models. CoRR, abs/2308.07269, 2023. find
  57. Yichi Zhang, Zhuo Chen, Yin Fang, Lei Cheng, Yanxi Lu, Fangming Li, Wen Zhang, and Huajun Chen. Knowledgeable preference alignment for llms in domain-specific question answering. CoRR, abs/2311.06503, 2023. find
  58. Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, and Huajun Chen. Detoxifying large language models via knowledge editing. CoRR, abs/2403.14472, 2024.
  59. Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. A comprehensive study o… find
  60. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. CoRR, abs/2305.13172, 2023. find
  61. Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeff Wu. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. CoRR, abs/2312.09390, 2023.
  62. Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. Augmented language models: a survey. CoRR, abs/2302.07842, 2023.
  63. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. CoRR, abs/2302.04761, 2023.
  64. Sheila McIlraith and Tran Cao Son. Adapting golog for composition of semantic web services. 2002. find
  65. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang,… — The Rise and Potential of Large Language Model Based Agents: A Survey
  66. Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. Agents: An open-source framework for autonomous language agents. CoRR, a… find
  67. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving AI tasks with chatgpt and its friends in huggingface. CoRR, abs/2303.17580, 2023.
  68. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: communicative agents for "mind" exploration of large scale language model society. CoRR, abs/2303.17760, 2023.
  69. Ian Horrocks, Peter F. Patel-Schneider, and Frank van Harmelen. From SHIQ and RDF to OWL: the making of a web ontology language. J. Web Semant., 1(1):7–26, 2003. find
  70. Michael R. Genesereth. Knowledge interchange format. In KR, pages 599–600. Morgan Kaufmann, 1991. find
  71. Grit Denker, Jerry R. Hobbs, David L. Martin, Srini Narayanan, and Richard J. Waldinger. Accessing information and services on the daml-enabled web. In SemWeb, volume 40 of CEUR Workshop Proceedings. CEUR-WS.org, 2001. find
  72. Yuqi Zhu, Shuofei Qiao, Yixin Ou, Shumin Deng, Ningyu Zhang, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. Knowagent: Knowledge-augmented planning for llm-based agents. CoRR, abs/2403.03101, 2024. find
  73. Jiaoyan Chen, Yuxia Geng, Zhuo Chen, Ian Horrocks, Jeff Z. Pan, and Huajun Chen. Knowledge-aware zero-shot learning: Survey and perspective. In IJCAI, pages 4366–4373. ijcai.org, 2021. find
  74. Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, Jiaqi Li, Xiaoze Liu, Jeff Z. Pan, Ningyu Zhang, and Huajun Chen. Knowledge graphs meet multi-modal learning: A comprehensive survey. CoRR, abs/2402.05391, 2024. — [arXiv:2402.05391](https://arxiv.org/abs/2402.05391) find
  75. Yuqi Zhu, Xiaohan Wang, Jing Chen, Shuofei Qiao, Yixin Ou, Yunzhi Yao, Shumin Deng, Huajun Chen, and Ningyu Zhang. Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities. CoRR, abs/2305.13168, 2023. find
  76. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. In IJCNLP (1), pages 675…
  77. Jun Gao, Huan Zhao, Changlong Yu, and Ruifeng Xu. Exploring the feasibility of chatgpt for event extraction. CoRR, abs/2303.03836, 2023.
  78. Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, Yong Jiang, and Wenjuan Han. Zero-shot information extraction via chatting with chatgpt. CoRR, abs/2302.10205, 2023.
  79. Honghao Gui, Jintian Zhang, Hongbin Ye, and Ningyu Zhang. Instructie: A chinese instruction-based information extraction dataset. CoRR, abs/2305.11527, 2023. — [arXiv:2305.11527](https://arxiv.org/abs/2305.11527) find
  80. Shuofei Qiao, Honghao Gui, Huajun Chen, and Ningyu Zhang. Making language models better tool learners with execution feedback. CoRR, abs/2305.13068, 2023. find
  81. Ruijie Wang, Zhiruo Zhang, Luca Rossetto, Florian Ruosch, and Abraham Bernstein. Nlqx-form: A language model-based question to SPARQL transformer. CoRR, abs/2311.07588, 2023. — [arXiv:2311.07588](https://arxiv.org/abs/2311.07588) find
  82. Apoorv Saxena, Aditay Tripathi, and Partha P. Talukdar. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In ACL, pages 4498– 4507. Association for Computational Linguistics, 2020.
  83. Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. Reasoning with language model prompting: A survey. In ACL (1), pages 5368–5393. Association for Computational Linguistics, 2023.
  84. Zezhong Xu, Wen Zhang, Peng Ye, Hui Chen, and Huajun Chen. Neural-symbolic entangled framework for complex query answering. In NeurIPS, 2022. find
  85. Wen Zhang, Jiaoyan Chen, Juan Li, Zezhong Xu, Jeff Z. Pan, and Huajun Chen. Knowledge graph reasoning with logics and embeddings: Survey and perspective. CoRR, abs/2202.07412, 2022. — [arXiv:2202.07412](https://arxiv.org/abs/2202.07412) find
  86. Jiarui Jin, Yangkun Wang, Kounianhua Du, Weinan Zhang, Zheng Zhang, David Wipf, Yong Yu, and Quan Gan. Inductive relation prediction using analogy subgraph embeddings. In ICLR. OpenReview.net, 2022. find
  87. Zezhong Xu, Peng Ye, Hui Chen, Meng Zhao, Huajun Chen, and Wen Zhang. Ruleformer: Context-aware rule mining over knowledge graph. In COLING, pages 2551–2560. International Committee on Computational Linguistics, 2022. find
  88. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Trans. Mach. Learn. Res.…
  89. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — Language Models are Few-Shot Learners
  90. Yanbin Wei, Qiushi Huang, Yu Zhang, and James T. Kwok. KICGPT: large language model with knowledge in context for knowledge graph completion. In EMNLP (Findings), pages 8667–8683. Association for Computational Linguistics, 2023. find
  91. Yichi Zhang, Zhuo Chen, Wen Zhang, and Huajun Chen. Making large language models perform better in knowledge graph completion. CoRR, abs/2310.06671, 2023. — [arXiv:2310.06671](https://arxiv.org/abs/2310.06671) find
  92. Wen Zhang, Yushan Zhu, Mingyang Chen, Yuxia Geng, Yufeng Huang, Yajing Xu, Wenting Song, and Huajun Chen. Structure pretraining and prompt tuning for knowledge graph transfer. In WWW, pages 2581–2590. ACM, 2023. find
  93. Chunkit Chan, Jiayang Cheng, Weiqi Wang, Yuxin Jiang, Tianqing Fang, Xin Liu, and Yangqiu Song. Chatgpt evaluation on sentence level relations: A focus on temporal, causal, and discourse relations. CoRR, abs/2304.14827, 2023.
  94. Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S. Yu, and Chuan Shi. Towards graph foundation models: A survey and beyond. CoRR, abs/2310.11829, 2023. — [arXiv:2310.11829](https://arxiv.org/abs/2310.11829) find
  95. Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. CoRR, abs/2310.13023, 2023.
  96. John F. Sowa. The challenge of knowledge soup. 2006. find
  97. Yoshua Bengio and Edward J. Hu. Scaling in the service of reasoning & model-based ml, 2023. find
  98. Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, and Ji-Rong Wen. Structgpt: A general framework for large language model to reason over structured data. CoRR, abs/2305.09645, 2023. find
  99. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In ICLR. OpenReview.net, 2023. find
  100. Ang Lv, Kaiyi Zhang, Yuhan Chen, Yulong Wang, Lifeng Liu, Ji-Rong Wen, Jian Xie, and Rui Yan. Interpreting key mechanisms of factual recall in transformer-based language models. CoRR, abs/2403.19521, 2024. — [arXiv:2403.19521](https://arxiv.org/abs/2403.19521) find
  101. David Ha and Jürgen Schmidhuber. World models. CoRR, abs/1803.10122, 2018. find
  102. Marvin Minsky. The emotion machine: Commonsense thinking, artificial intelligence, and the future of the human mind. Simon and Schuster, 2007. find
  103. Douglas B. Lenat, Ramanathan V. Guha, Karen Pittman, Dexter Pratt, and Mary Shepherd. CYC: toward programs with common sense. Commun. ACM, 33(8):30–49, 1990. find
  104. Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI, pages 4444–4451. AAAI Press, 2017. find
  105. Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: a core of semantic knowledge. In WWW, pages 697–706. ACM, 2007. find
  106. Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, and Christian Bizer. Dbpedia - A large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web, 6(2):167–195, 2015. — DBpedia – A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia
  107. Denny Vrandecic and Markus Krötzsch. Wikidata: a free collaborative knowledgebase. Commun. ACM, 57(10):78–85, 2014.
  108. Roberto Navigli and Simone Paolo Ponzetto. Babelnet: Building a very large multilingual semantic network. In ACL, pages 216–225. The Association for Computer Linguistics, 2010.
  109. Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. ATOMIC: an atlas of machine commonsense for if-then reasoning. In AAAI, pages 3027–3035. AAAI Press, 2019. find
  110. Hongming Zhang, Xin Liu, Haojie Pan, Haowen Ke, Jiefu Ou, Tianqing Fang, and Yangqiu Song. Aser: Towards large-scale commonsense knowledge acquisition via higher-order selectional preference over eventualities. Artificial Intelligence, 309:103740, 2022. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2023-large-knowledge-model-perspectives-challenges/