Elamur Research⌕
☰
home / papers / 2018 / A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network

A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network

Summary

Knowledge bases such as Freebase or DBpedia store facts as (head, relation, tail) triples, but they lack many true facts. Models are therefore needed to predict missing links. The authors propose ConvKB, an embedding model that applies a convolutional neural network to whole triples. The head, relation and tail embeddings of a triple are stacked as columns of a k×3 matrix. Several 1×3 filters slide across its rows, so each filter compares the same dimension across all three embeddings. The resulting feature maps are joined and multiplied with a weight vector to give one triple score. ConvE convolves only reshaped head and relation embeddings. ConvKB instead keeps the translation intuition of TransE, which it contains as a special case. ConvKB is trained with Adam on a logistic loss and initialized from TransE embeddings. It was tested on WN18RR and FB15k-237, two benchmarks without the reversible-relation shortcut. It achieved the best mean rank and Hits@10 on WN18RR and the best MRR and Hits@10 on FB15k-237. This shows that a simple CNN over triples is a strong approach to link prediction.

Key points

  • Plain TransE as a special case: ConvKB reduces exactly to TransE when it uses one fixed filter [1, 1, −1], zero bias, an absolute-value or squared activation and w = 1.
  • Benchmark datasets:
    • WN18RR has 40,943 entities and 11 relations.
    • FB15k-237 has 14,541 entities and 237 relations.
    • Both are harder subsets of WN18 and FB15k, with the easy reversible relations removed.
  • WN18RR results: ConvKB reached MR 2554, MRR 0.248 and Hits@10 of 52.5%. Its own TransE baseline scored MR 3384 and 50.1%.
  • FB15k-237 results:
    • ConvKB reached MRR 0.396 and Hits@10 of 51.7%.
    • ConvE scored 0.316 / 49.1%, and TransE scored 0.294 / 46.5%.
    • The gain in MRR over TransE is about 34% relative.
  • Setup and TransE as a baseline:
    • Evaluation used the "Filtered" ranking protocol with MR, MRR and Hits@10.
    • The authors show that TransE, which was omitted in the ConvE paper, is a strong baseline.
    • On WN18RR, TransE even beats ConvE on MR and Hits@10.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

79 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 77 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-2053

@inproceedings{nguyen2018novel,
  title = {{A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network}},
  author = {Nguyen, Dai Quoc and Nguyen, Tu Dinh and Nguyen, Dat Quoc and Phung, Dinh},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-2053},
  url = {https://doi.org/10.18653/v1/n18-2053}
}

References

5 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pages 1247–1250. find
  2. Antoine Bordes, Nicolas Usunier, Alberto Garcı́a-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems 26, pages 2787–2795. find
  3. Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning Structured Embeddings of Knowledge Bases. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, pages 301–306. find
  4. Liwei Cai and William Yang Wang. 2018. KBGAN: Adversarial Learning for Knowledge Graph Embeddings. In Proceedings of The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page to appear. find
  5. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537. \#\# Conclusion find our card

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2018-novel-embedding-model-knowledge-base/