Elamur Research⌕
☰
home / papers / 2019 / RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space

Paper · 2019 · Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie et al.

Summary

The paper addresses the task of knowledge graph completion, that is, predicting missing links in fact bases such as Freebase or WordNet. To this end, entities and relations are represented as vectors. The authors argue that the quality of such a model depends on whether it can capture three typical properties of relations: symmetry or antisymmetry, the existence of an inverse relation, and composition. Previous models (TransE, DistMult, ComplEx and others) cover only some of these properties. The new model, RotatE, represents entities as complex vectors and each relation as a rotation of each coordinate of the head entity. The modulus of the relation components equals one, so only the phase changes. This simple mechanism suffices to formally describe all three properties, while time and memory costs grow linearly. For training, the authors also propose self-adversarial negative sampling. It assigns greater weight to false triples based on the model's current scores, and it can be applied to other models. RotatE outperforms the state-of-the-art methods of its time on several standard datasets. It has thus become a simple, scalable and theoretically grounded baseline model for knowledge graphs.

Key points

  • Model idea: a relation acts as element-wise multiplication by a complex number with modulus 1. Symmetric relations have phases of 0 or π, inverse pairs have conjugate vectors, and composition corresponds to the sum of phases. TransE is a limiting case of RotatE, and TorusE is a special case with a fixed modulus.
  • Evaluation data: FB15k, WN18, FB15k-237, WN18RR, YAGO3-10 (123,182 entities, 37 relations) and Countries, a dataset created specifically to test composition.
  • Results (mean MRR over three runs with different seeds): FB15k ≈ 0.797, WN18 ≈ 0.949, FB15k-237 ≈ 0.337, WN18RR ≈ 0.477. On YAGO3-10, MRR is 0.495 versus 0.44 for ConvE, and Hits@10 is 0.670. The variance between runs does not exceed ±0.001.
  • Ablation on FB15k-237: self-adversarial sampling improves MRR for both RotatE and TransE (for RotatE, from 0.297 to 0.338). The negative-sampling-based loss yields a gain only for RotatE.
  • Resource: the code is published in the DeepGraphLearning/KnowledgeGraphEmbedding repository on GitHub. The embedding dimension in the paper is 500–1000, and the number of negative samples ranges from 64 to 1024 depending on the dataset.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China244 works · 39,359 citations🎓Université de MontréalMontreal · Canada68 works · 79,382 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Knowledge graphs & ontologies · 192 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Knowledge graph · 49 works

Cite

Cite as Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie and Jian Tang. 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. arXiv:1902.10197.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1902.10197 · arXiv 1902.10197

@misc{sun2019rotate,
  title = {{RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space}},
  author = {Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie and Jian Tang},
  year = {2019},
  doi = {10.48550/arxiv.1902.10197},
  eprint = {1902.10197},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1902.10197}
}

References

34 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp. 1247–1250. find
  2. Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio, et al. Learning structured embeddings of knowledge bases. In AAAI, volume 6, pp. 6, 2011. find
  3. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pp. 2787–2795, 2013.
  4. Guillaume Bouchard, Sameer Singh, and Theo Trouillon. On approximate reasoning capabilities of low-rank vector spaces. AAAI Spring Syposium on Knowledge Representation and Reasoning (KRR): Integrating Symbolic and Neural Approaches, 2015. find
  5. Liwei Cai and William Yang Wang. Kbgan: Adversarial learning for knowledge graph embeddings. arXiv preprint arXiv:1711.04071, 2017. find
  6. Rajarshi Das, Arvind Neelakantan, David Belanger, and Andrew McCallum. soning over entities, relations, and text using recurrent neural networks. arXiv:1607.01426, 2016. find
  7. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. Convolutional 2d knowledge graph embeddings. arXiv preprint arXiv:1707.01476, 2017. find
  8. Takuma Ebisu and Ryutaro Ichise. Toruse: Knowledge graph embedding on a lie group. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1819–1826. find
  9. Kelvin Guu, John Miller, and Percy Liang. Traversing knowledge graphs in vector space. arXiv preprint arXiv:1506.01094, 2015. find
  10. Yanchao Hao, Yuanzhe Zhang, Kang Liu, Shizhu He, Zhanyi Liu, Hua Wu, and Jun Zhao. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon… find
  11. Katsuhiko Hayashi and Masashi Shimbo. On the equivalence of holographic and complex embeddings for link prediction. arXiv preprint arXiv:1702.05563, 2017. find
  12. Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. Learning to represent knowledge graphs with gaussian embedding. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 623–632. ACM, 2015. find
  13. Rudolf Kadlec, Ondrej Bajgar, and Jan Kleindienst. Knowledge base completion: Baselines strike back. arXiv preprint arXiv:1705.10744, 2017. find
  14. Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  15. Ni Lao, Tom Mitchell, and William W Cohen. Random walk inference and learning in a large scale knowledge base. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 529–539. Association for Computational Linguistics, 2011. find
  16. Yankai Lin, Zhiyuan Liu, Huanbo Luan, Maosong Sun, Siwei Rao, and Song Liu. Modeling relation paths for representation learning of knowledge bases. arXiv preprint arXiv:1506.00379, 2015a.
  17. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. Learning entity and relation embeddings for knowledge graph completion. In AAAI, volume 15, pp. 2181–2187, 2015b. find
  18. Farzaneh Mahdisoltani, Joanna Biega, and Fabian M Suchanek. Yago3: A knowledge base from multilingual wikipedias. In CIDR, 2013.
  19. Chains of rea-arXiv preprint Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pp. 3111–3119, 2013.
  20. George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11): 39–41, 1995.
  21. Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. Compositional vector space models for knowledge base completion. arXiv preprint arXiv:1504.06662, 2015.
  22. Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121, 2017.
  23. Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. Stranse: a novel embedding model of entities and relationships in knowledge bases. arXiv preprint arXiv:1606.08140, 2016.
  24. Maximilian Nickel, Lorenzo Rosasco, Tomaso A Poggio, et al. Holographic embeddings of knowledge graphs. In AAAI, volume 2, pp. 3–2, 2016. find
  25. Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving. In Advances in Neural Information Processing Systems, pp. 3788–3800, 2017. find
  26. Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, pp. 697–706. ACM, 2007. find
  27. Kristina Toutanova and Danqi Chen. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, pp. 57–66, 2015.
  28. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In International Conference on Machine Learning, pp. 2071–2080, 2016.
  29. Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. Knowledge graph embedding by translating on hyperplanes. In AAAI, volume 14, pp. 1112–1119, 2014. find
  30. Chenyan Xiong, Russell Power, and Jamie Callan. Explicit semantic ranking for academic search via knowledge graph embedding. In Proceedings of the 26th international conference on world wide web, pp. 1271–1279. International World Wide Web Conferences Steering Committee, 2017. find
  31. Bishan Yang and Tom Mitchell. Leveraging knowledge bases in lstms for improving machine reading. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pp. 1436–1446, 2017. find
  32. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575, 2014.
  33. Fan Yang, Zhilin Yang, and William W Cohen. Differentiable learning of logical rules for knowledge base completion. CoRR, abs/1702.08367, 2017. find
  34. Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 353–362. ACM, 2016. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sun-2019-rotate-knowledge-graph-embedding-relational/