Elamur Research
☰
home / papers / 2019 / TuckER: Tensor Factorization for Knowledge Graph Completion

TuckER: Tensor Factorization for Knowledge Graph Completion

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1901.09590

Summary

Knowledge graphs store facts as "subject – relation – object" triples but are almost always incomplete. Methods are therefore needed that recover missing links, i.e., that solve the link prediction task. The authors propose TuckER, a linear model. It represents the graph as a three-dimensional binary tensor and factorizes it using Tucker decomposition. The rows of the factor matrices serve as embeddings of entities and relations, while a compact core tensor determines how they interact. The core is shared across all relations, so the model transfers knowledge between them (multi-task learning), and its number of parameters grows only linearly with the number of entities and relations. The authors prove that TuckER is a fully expressive model and derive a bound on the embedding dimensionality that guarantees this. They also show that RESCAL, DistMult, ComplEx and SimplE are special cases of TuckER. On four standard datasets the model outperforms previous approaches, including the non-linear convolutional ConvE and HypER. The value of the work lies in showing that a simple and mathematically transparent model becomes a strong baseline for more complex approaches.

Key points

  • Method: the score of a triple is the product of the core tensor W with the subject, relation and object embeddings along the three modes. A logistic sigmoid is then applied. An entity has the same embedding both in the subject role and in the object role.
  • Results: on WN18RR, MRR 0.470 (Hits@1 0.443); on FB15k-237, MRR 0.358 (Hits@10 0.544); on FB15k, MRR 0.795; on WN18, MRR 0.953.
  • Efficiency: on WN18RR, TuckER has about 9.4M parameters versus about 16.4M for ComplEx and SimplE, yet it is still more accurate. On FB15k-237, with dimensionality 20 (about 300K parameters), it nearly matches ComplEx and SimplE with dimensionality 200 (about 6M).
  • Role of the shared core: TuckER's margin is largest on datasets with many relations. On FB15k, in Hits@1 it outperforms ComplEx by 14% and SimplE by 8%.
  • Comparison with ComplEx-N3: this is another linear model with multi-task learning. Its published results were not compared, because it has far more parameters. When the number of parameters was equalized, both models showed comparable performance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

143 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 141 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1522 · arXiv 1901.09590

@misc{balazevic2019tucker,
  title = {{TuckER: Tensor Factorization for Knowledge Graph Completion}},
  author = {Balažević, Ivana and Allen, Carl and Hospedales, Timothy M.},
  year = {2019},
  doi = {10.18653/v1/d19-1522},
  eprint = {1901.09590},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d19-1522}
}

References

31 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ivana Balažević, Carl Allen, and Timothy M Hospedales. 2019. Hypernetwork Knowledge Graph Embeddings. In International Conference on Artificial Neural Networks. find
  2. Hedi Ben-Younes, Rémi Cadene, Matthieu Cord, and Nicolas Thome. 2017. MUTAN: Multimodal Tucker Fusion for Visual Question Answering. In International Conference on Computer Vision. find
  3. Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems. find
  4. Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, and Andrew McCallum. 2018. Go for a Walk and Arrive at the Answer: Reasoning over Paths in Knowledge Bases Using Reinforcement Learning. In International Conference on Learning Representa… find
  5. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D Knowledge Graph Embeddings. In Association for the Advancement of Artificial Intelligence. find
  6. Takuma Ebisu and Ryutaro Ichise. 2018. TorusE: Knowledge Graph Embedding on a Lie Group. In Association for the Advancement of Artificial Intelligence. find
  7. Jun Feng, Minlie Huang, Mingdong Wang, Mantong Zhou, Yu Hao, and Xiaoyan Zhu. 2016. Knowledge Graph Embedding by Flexible Translation. In Principles of Knowledge Representation and Reasoning. find
  8. Frank L Hitchcock. 1927. The Expression of a Tensor or a Polyadic as a Sum of Products. Journal of Mathematics and Physics, 6(1-4):164–189. find
  9. Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In International Conference on Machine Learning. find
  10. Seyed Mehran Kazemi and David Poole. 2018. SimplE Embedding for Link Prediction in Knowledge Graphs. In Advances in Neural Information Processing Systems. find
  11. Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations. find
  12. Tamara G Kolda and Brett W Bader. 2009. Tensor Decompositions and Applications. SIAM review, 51(3):455–500. find
  13. Pieter M Kroonenberg and Jan De Leeuw. 1980. Principal Component Analysis of Three-Mode Data by Means of Alternating Least Squares Algorithms. Psychometrika, 45(1):69–97. find
  14. Timothée Lacroix, Nicolas Usunier, and Guillaume Obozinski. 2018. Canonical Tensor Decomposition for Knowledge Base Completion. In International Conference on Machine Learning. find
  15. Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical Inference for Multi-relational Embeddings. In International Conference on Machine Learning. find
  16. Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson. 2016. STransE: a Novel Embedding Model of Entities and Relationships in Knowledge Bases. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
  17. Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A Three-Way Model for Collective Learning on Multi-Relational Data. In International Conference on Machine Learning. find
  18. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic Differentiation in PyTorch. In NIPS-W. find
  19. Aaron Schein, Mingyuan Zhou, David Blei, and Hanna Wallach. 2016. Bayesian Poisson Tucker Decomposition for Learning the Structure of International Relations. In International Conference on Machine Learning. find
  20. Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling Relational Data with Graph Convolutional Networks. In European Semantic Web Conference. find
  21. Yelong Shen, Jianshu Chen, Po-Sen Huang, Yuqing Guo, and Jianfeng Gao. 2018. M-Walk: Learning to Walk over Graphs using Monte Carlo Tree Search. In Advances in Neural Information Processing Systems. find
  22. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 15(1):1929–1958. find
  23. Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In International Conference on Learning Representations. find
  24. Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing Text for Joint Embedding of Text and Knowledge Bases. In Empirical Methods in Natural Language Processing. find
  25. Théo Trouillon, Christopher R Dance, Éric Gaussier, Johannes Welbl, Sebastian Riedel, and Guillaume Bouchard. 2017. Knowledge Graph Completion via Complex Tensor Factorization. Journal of Machine Learning Research, 18(1):4735–4772. find
  26. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In International Conference on Machine Learning. find
  27. Ledyard R Tucker. 1964. The Extension of Factor Analysis to Three-Dimensional Matrices. Contributions to Mathematical Psychology, 110119. find
  28. Ledyard R Tucker. 1966. Some Mathematical Notes on Three-Mode Factor Analysis. Psychometrika, 31(3):279–311. find
  29. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In International Conference on Learning Representations. find
  30. Fan Yang, Zhilin Yang, and William W Cohen. 2017. Differentiable Learning of Logical Rules for Knowledge Base Reasoning. In Advances in Neural Information Processing Systems. find
  31. Yongxin Yang and Timothy Hospedales. 2017. Deep Multi-task Representation Learning: A Tensor Factorisation Approach. In International Conference on Learning Representations. find