Elamur Research⌕
☰
home / papers / 2026 / Linguistic Initialization for Inductive Reasoning in Heterogeneous Knowledge Graphs

Linguistic Initialization for Inductive Reasoning in Heterogeneous Knowledge Graphs

Paper · 2026 · Daniele Pasquini, Danilo Croce and Roberto Basili

Summary

Graph neural networks for knowledge graph completion usually start from random or identifier-based node embeddings. They must then infer both what a node means and how it connects, which fails for sparse or cold-start entities. Such models tend to fall back on popular hub-based defaults. Pasquini, Croce and Basili address this for inductive link prediction in heterogeneous knowledge graphs. Before any message passing, each node gets a short text that joins its own description with a verbalized summary of its local neighborhood. A pre-trained language model embeds this text, and the vectors become the starting states of a relation-aware, multi-head attention GNN. The GNN uses type-specific projections and a residual fallback, so isolated nodes keep their text-derived meaning. The authors test the approach under several class-imbalance regimes and a strict entity-to-entity cold-start split. Text-grounded initialization consistently beats random initialization and largely removes the drop in performance on low-degree nodes. This matters because language-model knowledge can enter existing heterogeneous GNN pipelines with almost no architectural change.

Key points

  • Method: a dual-view textual initialization (intrinsic description plus neighborhood context) feeds relation-aware attention layers. The relation types follow a Freebase-style schema, such as medicine.disease.risk_factors and book.written_work.author.
  • Relation-label prediction: with balanced supervision, Random reaches F1=.95 and Text reaches .97. At 1:100 imbalance, Random falls to .60 while Text keeps .72. A text-only MLP without graph propagation gets just .29.
  • Hard set (degree ≤ 1), Head relations: recall rises from .68 to .88 and F1 from .81 to .93 with text initialization.
  • Degree-stratified recall (Low/Medium/High): Random scores .67/.72/.86, while Text scores .88/.87/.90. The Low–High gap shrinks to about .02, and low-degree nodes gain roughly +.20.
  • Overall pattern: structure dominates in dense regions, while language priors stabilize sparse regions and help tell apart predicates with similar structural footprints.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Rome Tor VergataRome · Italy15 works · 1,321 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksKnowledge graphs & ontologies · 192 works

Cite

Cite as Daniele Pasquini, Danilo Croce and Roberto Basili. 2026. Linguistic Initialization for Inductive Reasoning in Heterogeneous Knowledge Graphs. Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2cxo43n6o593

@inproceedings{pasquini2026linguistic,
  title = {{Linguistic Initialization for Inductive Reasoning in Heterogeneous Knowledge Graphs}},
  author = {Daniele Pasquini, Danilo Croce and Roberto Basili},
  booktitle = {Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26},
  year = {2026},
  doi = {10.63317/2cxo43n6o593},
  url = {https://doi.org/10.63317/2cxo43n6o593}
}

References

33 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 1263–1272. JMLR.org.
  2. Saurabh Bajaj, Hojae Son, Juelin Liu, Hui Guan, and Marco Serafini. 2024. Graph neural network training systems: A performance comparison of full-graph and mini-batch. Proc. VLDB Endow., 18(4):1196–1209. find
  3. Gaole He, Yunshi Lan, Jing Jiang, Wayne Xin Zhao, and Ji-Rong Wen. 2021. Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In WSDM. find
  4. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26.
  5. Tao He, Ming Liu, Yixin Cao, Zekun Wang, Zihao Zheng, and Bing Qin. 2024. Exploring & exploiting high-order graph structure for sparse knowledge graph completion. Front. Comput. Sci., 19(2). find
  6. Benjamin Paul Chamberlain, Sergey Shirobokov, Emanuele Rossi, Fabrizio Frasca, Thomas Markovich, Nils Hammerla, Michael M. Bronstein, and Max Hansmire. 2023. Graph neural networks for link prediction with subgraph sketching. find
  7. Zirui Liu, Chen Shengyuan, Kaixiong Zhou, Daochen Zha, Xiao Huang, and Xia Hu. 2023. RSC: Accelerate graph neural networks training via randomized sparse computations. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pag… find
  8. Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, and Jiliang Tang. 2024. Exploring the potential of large language models (llms)in learning on graphs. SIGKDD Explor. Newsl., 25(2):42–61. find
  9. Yihong Ma, Yijun Tian, Nuno Moniz, and Nitesh V. Chawla. 2025. Class-imbalanced learning on graphs: A survey. ACM Comput. Surv., 57(8). find
  10. Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. 2017. Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach . In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2233–2241, Los Alamitos, CA, USA. IEEE Computer… find
  11. Yoonhyuk Choi, Jiho Choi, Taewook Ko, and Chong-Kwon Kim. 2025. Mitigating overfitting in graph neural networks via feature and hyperplane perturbation. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM ’25, page 50–59, New York, NY, USA. Association f… find
  12. Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. Graph retrieval-augmented generation: A survey.
  13. Kenneth Church and Yuchen Bian. 2021. Data collection vs. knowledge graph completion: What is needed to improve coverage? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6210– 6215, Online and Punta Cana, Dominican Republic. Association for Computatio… find
  14. Tieyun Qian, Yile Liang, Qing Li, and Hui Xiong. 2023. Attribute Graph Neural Networks for Strict Cold Start Recommendation : Extended Abstract . In 2023 IEEE 39th International Conference on Data Engineering (ICDE), pages 3783–3784, Los Alamitos, CA, USA. IEEE Computer Society. find
  15. Enyan Dai, Wei Jin, Hui Liu, and Suhang Wang. 2022. Towards robust graph neural networks for noisy graphs with sparse labels. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, WSDM ’22, page 181–191, New York, NY, USA. Association for Computing Machinery. find
  16. Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European semantic web conference, pages 593–607. Springer. find
  17. Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2024. Talk like a graph: Encoding graphs for large language models. In The Twelfth International Conference on Learning Representations. find
  18. Marco Serafini and Hui Guan. 2021. Scalable graph neural network training: The case for sampling. SIGOPS Oper. Syst. Rev., 55(1):68–76. find
  19. Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. find
  20. Jaeyong Song, Hongsun Jang, Hunseong Lim, Jaewon Jung, Youngsok Kim, and Jinho Lee. 2024. for Computational Linguistics (Volume 1: Long Papers), pages 2526–2547, Vienna, Austria. Association for Computational Linguistics. find
  21. Granndis: Fast distributed graph neural network training framework for multi-server clusters. In Proceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques, PACT ’24, page 91–107, New York, NY, USA. Association for Computing Machinery. find
  22. Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016. Representation learning of knowledge graphs with entity descriptions. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). find
  23. Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S. Yu, and Tianyi Wu. 2011. Pathsim: Meta path-based top-K similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment, 4(11):992–1003. find
  24. Bishan Yang, Wen tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases.
  25. Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S. Yu, and Tianyi Wu. 2022. Heterogeneous information networks: the past, the present, and the future. In Proceedings of the 48th International Conference on Very Large Data Bases (VLDB), volume 15, pages 3807–3811. find
  26. Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. Kg-bert: Bert for knowledge graph completion. find
  27. Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. CoRR, abs/1902.10197.
  28. Komal Teru, Etienne Denis, and Will Hamilton. 2020. Inductive relation prediction by subgraph reasoning. In International conference on machine learning, pages 9448–9457. PMLR. find
  29. Robert Dadashi et al. Thomas Mesnard, Cassidy Hardin. 2024. Gemma: Open models based on gemini research and technology. find
  30. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. 2017. Graph attention networks. stat, 1050(20):10–48550. find
  31. Liang Wang, Wei Zhao, Zhuoyu Wei, and Jingming Liu. 2022. SimKGC: Simple contrastive knowledge graph completion with pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4281–4294, Dublin, Ireland. Asso… find
  32. Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019. Heterogeneous graph attention network. In The world wide web conference, pages 2022–2032. find
  33. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Griffin Thomas Adams, Jeremy Howard, and Iacopo Poli. 2025. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory e…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pasquini-2026-linguistic-initialization-inductive-reasoning-heterogeneous/