Elamur Research⌕
☰
home / papers / 2016 / Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks

Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks

Paper · 2016 · Matthew Francis-Landau, Greg Durrett and Dan Klein

Summary

The paper tackles entity linking, specifically mentions whose correct target depends on context (e.g., Germany as a country, a government, or a football team). Rather than relying on collective linking or hand-tuned tf-idf heuristics, the authors learn topic representations with convolutional neural networks. Separate CNNs encode three source-side text spans (the mention, its nearby context, the full document) and two entity-side spans (the Wikipedia title and article body). Cosine similarities between every source/target pair become dense features. These features are added to the sparse log-linear system of Durrett and Klein (2014), which uses a latent query variable to choose which part of a mention to resolve against Wikipedia link counts. The whole model is trained end-to-end with Adadelta. On ACE, CoNLL-YAGO, WP and a Wikipedia dataset, the combined model beats DK2014 and AIDA-LIGHT. The results show that learned semantic similarity and sparse indicator features complement each other. A further finding is that using several context granularities works better than using any single one.

Key points

  • Each CNN uses filters of width ℓ = 5 and k = 150 filter rows, with ReLU and sum pooling. Six cosine-similarity features feed a logistic layer alongside the sparse features f_Q and f_E. Query generation yields about 9 candidate queries per mention.
  • Full model accuracy is 89.9 on ACE (vs. 79.6 for DK2014) and 85.5 on CoNLL (vs. 84.8 for AIDA-LIGHT). It also scores 90.7 on WP and 82.2 on Wikipedia.
  • CNN features alone outperform sparse features alone on ACE, CoNLL and WP, though not on Wikipedia (75.7 vs. 81.5). Combining the two gives the best results on every dataset.
  • Using only the document/article pair or only the mention/title pair does worse than using all CNN pairs.
  • Embeddings come from word2vec trained on Wikipedia (300 dimensions, window 21) and are kept frozen during training. On dev sets they beat Google News vectors, for example 89.5 vs. 87.5 on ACE, because Google News vectors have more out-of-vocabulary words. Code: github.com/matthewfl/nlp-entity-convnet.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California, BerkeleyBerkeley · United States175 works · 227,440 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Named entity recognition · 200 works

Cite

Cite as Matthew Francis-Landau, Greg Durrett and Dan Klein. 2016. Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1150

@inproceedings{francislandau2016capturing,
  title = {{Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks}},
  author = {Matthew Francis-Landau, Greg Durrett and Dan Klein},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1150},
  url = {https://doi.org/10.18653/v1/n16-1150}
}

References

20 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Luisa Bentivogli, Pamela Forner, Claudio Giuliano, Alessandro Marchetti, Emanuele Pianta, and Kateryna Tymoshenko. 2010. find
  2. Extending English ACE 2005 Corpus Annotation with Ground-truth Links to Wikipedia. In Proceedings of the Workshop on The People’s Web Meets NLP: Collaboratively Constructed Semantic Resources. find
  3. Xiao Cheng and Dan Roth. 2013. Relational Inference for Wikification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
  4. Silviu Cucerzan. 2007. Large-Scale Named Entity Disambiguation Based on Wikipedia Data. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). find
  5. Li Dong, Furu Wei, Ming Zhou, and Ke Xu. 2015. Question Answering over Freebase with Multi-Column Convolutional Neural Networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL) and the 7th International Joint Conference on Natural Language Processing… find
  6. Greg Durrett and Dan Klein. 2014. A Joint Model for Entity Analysis: Coreference, Typing, and Linking. In Transactions of the Association for Computational Linguistics (TACL).
  7. Angela Fahrni and Michael Strube. 2014. A latent variable model for discourse-aware concept and entity disambiguation. In Gosse Bouma and Yannick Parmentier 0001, editors, Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2014, April 26… find
  8. Zhengyan He, Shujie Liu, Yang Song, Mu Li, Ming Zhou, and Houfeng Wang. 2013. Efficient collective entity linking with stacking. In EMNLP, pages 426– 435. find
  9. ACL. Tom Heath and Christian Bizer. 2011. Linked Data: Evolving the Web into a Global Data Space. Morgan & Claypool, 1st edition. Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum. 2011. Robust Disa… find
  10. In Proceedings of the Association for Computational Linguistics (ACL). Heng Ji and Ralph Grishman. 2011. Knowledge Base Population: Successful Approaches and Challenges. In Proceedings of the Association for Computational Linguistics (ACL). find
  11. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 655–665, Baltimore, Maryland, June. Association for Computa…
  12. Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
  13. Omer Levy and Yoav Goldberg. 2014. Dependency-Based Word Embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL). find
  14. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems (NIPS) 26, pages 3111–3119.
  15. David Milne and Ian H. Witten. 2008. Learning to Link with Wikipedia. In Proceedings of the Conference on Information and Knowledge Management (CIKM). find
  16. Dat Ba Nguyen, Johannes Hoffart, Martin Theobald, and Gerhard Weikum. 2014. AIDA-light: High-Throughput Named-Entity Disambiguation. In Proceedings of the Workshop on Linked Data on the Web co-located with the 23rd International World Wide Web Conference (WWW). find
  17. NIST. 2005. The ACE 2005 Evaluation Plan. In NIST. find
  18. Lev Ratinov, Dan Roth, Doug Downey, and Mike Anderson. 2011. Local and Global Algorithms for Disambiguation to Wikipedia. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL), pages 1375–1384. Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire… find
  19. Yaming Sun, Lei Lin, Duyu Tang, Nan Yang, Zhenzhou Ji, and Xiaolong Wang. 2015. Modeling Mention, Context and Entity with Neural Networks for Entity Disambiguation. In Proceedings of the International Joint Conference on Artificial Intelligence (IJ-CAI), pages 1333–1339. find
  20. Matthew D. Zeiler. 2012. AdaDelta: An Adaptive Learning Rate Method. CoRR, abs/1212.5701.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/landau-2016-capturing-semantic-similarity-entity-linking/