Observed versus latent features for knowledge base and text inference

Summary

The paper addresses automatic knowledge base completion: given known "entity–relation–entity" triples, a model must find missing links. The authors compare two approaches. The first is latent feature models, i.e., vector representations of entities and relations (model E, DistMult). The second is a very simple observed feature model. It looks only at direct links between a pair of candidates and at how often a particular entity occupies a given position in a relation. On the popular benchmarks FB15K and WN18, this simple model clearly outperforms the latent ones. The reason lies in the redundancy of these graphs: they contain many near-duplicate and inverse relations. The authors therefore created a more challenging dataset, FB15KSelected, from which such relations were removed, and on it the latent models are stronger. They then added textual relations from the ClueWeb12 corpus to the graph. The best result was achieved by a joint model combining both types of features. The paper concludes that standard benchmarks can yield inflated scores and that web text substantially helps in completing knowledge bases.

Key points

Latent models: model E (similar to the universal schema approach) and DistMult (a diagonal bilinear variant of RESCAL). Observed features: a simplified analogue of PRA that considers only paths of length one. All models share the same loss function: negative log-likelihood with L2 regularization.
On FB15KSelected without text, the best observed feature model achieves an MRR of 23.5, while the E+DistMult combination achieves 26.6.
Textual relations were extracted from 200 million parsed ClueWeb12 sentences with Freebase annotations (FACC1). Each relation is represented by a lexicalized dependency path. After filtering, 25,000 textual relations remained, yielding 6.6 million new edges in the graph. Textual mentions exist for only 23.3% of test entity pairs.
If textual and KB relations are weighted equally in the loss function (τ = 1), there is no benefit. Reducing the weight of text to τ = 0.1 raises the MRR of the E+DistMult model to 27.4.
The observed feature model with text increases MRR on triples with mentions from 19.3 to 39.6. Combining latent and observed features yields an overall MRR of 29.3 and HITS@10 of 46.2.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics Knowledge graphs & ontologies

Cite

Cite as Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-4007

@inproceedings{toutanova2015observed,
  title = {{Observed versus latent features for knowledge base and text inference}},
  author = {Kristina Toutanova and Danqi Chen},
  booktitle = {Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality},
  year = {2015},
  doi = {10.18653/v1/w15-4007},
  url = {https://doi.org/10.18653/v1/w15-4007}
}

References

21 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. Springer. find
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250. ACM. find
Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Conference on Artificial Intelligence, number EPFL-CONF-192344. find
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems (NIPS). find
Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. 2014. Typed tensor decomposition of knowledge bases for relation extraction. In Empirical Methods in Natural Language Processing (EMNLP). find
Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In International Conference on Knowledge Discovery and Data Mining (KDD). find
Evgeniy Gabrilovich, Michael Ringgaard, and Amarnag Subramanya. 2013. FACC1: Freebase annotation of ClueWeb corpora, Version 1 (release date 2013-06-26, format version 1, correction level 0). find
Matt Gardner, Partha Pratim Talukdar, Bryan Kisiel, and Tom Mitchell. 2013. Improving learning and inference in a large knowledge-base using latent syntactic cues. In Empirical Methods in Natural Language Processing (EMNLP). find
Matt Gardner, Partha Talukdar, Jayant Krishnamurthy, and Tom Mitchell. 2014. Incorporating vector space similarity in random walk inference over knowledge bases. In Empirical Methods in Natural Language Processing (EMNLP). find
Ni Lao and William W Cohen. 2010. Relational retrieval using a combination of path-constrained random walks. Machine learning. find
Ni Lao, Tom Mitchell, and William W Cohen. 2011. Random walk inference and learning in a large scale knowledge base. In Empirical Methods in Natural Language Processing (EMNLP). find
Ni Lao, Amarnag Subramanya, Fernando Pereira, and William W Cohen. 2012. Reading the web with learned syntactic-semantic inference rules. In Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP/CoNLL). find
Dong C Liu and Jorge Nocedal. 1989. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1-3):503–528. find
Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 809–816. find
Maximilian Nickel, Xueyan Jiang, and Volker Tresp. 2014. Reducing the rank in relational factorization models by including observable patterns. In Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 1179–1187.… find
Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2015. A review of relational machine learning for knowledge graphs: From multi-relational link prediction to automated knowledge graph construction. arXiv preprint arXiv:1503.00759. find
Sebastian Riedel, Limin Yao, Benjamin M. Marlin, and Andrew McCallum. 2013. Relation extraction with matrix factorization and universal schemas. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). find
Martin Riedmiller and Heinrich Braun. 1993. A direct adaptive method for faster backpropagation learning: The rprop algorithm. In Neural Networks, 1993., IEEE International Conference on, pages 586–591. IEEE. find
Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web, pages 697–706. ACM. find
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pages 1112–1119. find
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In International Conference on Learning Representations (ICLR). our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/toutanova-2015-observed-versus-latent-features-knowledge/