Elamur Research⌕
☰
home / papers / 2010 / Entity Disambiguation for Knowledge Base Population

Entity Disambiguation for Knowledge Base Population

Summary

The paper tackles entity linking: mapping a person, organization or geo-political mention in a document to its entry in a large knowledge base, or returning NIL when no entry exists. The authors identify three difficulties: inconsistent name forms, ambiguous names, and entities missing from the KB. Their system has two stages. First, a KB-independent candidate selection step gathers plausible entries using exact and partial title matches, acronym matching, aliases and string-similarity measures, and it is indexed so it runs quickly. Second, a supervised ranker scores the candidates with a broad feature set. NIL is treated as one more candidate to rank, so the model learns directly when to abstain instead of relying on a hand-tuned threshold. They evaluate on TAC-KBP 2009 with a Wikipedia-derived KB of about 818,000 entries and reach roughly 80% accuracy, the best reported macro-averaged score. On Cucerzan's MSNBC news data they reach about 95%. The work matters because it does not depend on Wikipedia markup, carries over to other curated KBs, and models absence from the KB explicitly.

Key points

  • KB size and candidate recall. The KB holds 818,000 entities, 35% of which are PER, ORG or GPE. Candidate selection recall was 98.8% on training data and 98.6% on the TAC-KBP test set, and the candidate lists it produced were roughly 1,000 to 10,000 times shorter than the full KB.
  • Selection filters. Candidates come from:
    • exact title matches,
    • titles containing the mention or contained in it,
    • initials matching the title,
    • known aliases,
    • string similarity: character Dice > 0.9, skip-bigram Dice > 0.6, Hamming distance ≤ 2,
    • (Wikipedia-specific) the top 20 Google results.
  • Speed. Indexing cut selection time from about two minutes per query (naive linear scan) to under two seconds. Dice computation used over 95% of processing time but was needed for fewer than 4% of training queries. On the MSNBC data the median time was 1.98 s per query.
  • TAC-KBP results. Micro-averaged accuracy with all features was 0.7984. Candidate counts per query had a mean of 76, a median of 15 and a maximum of 2772 (for "Texas"). The ablation suggests that dropping the feature combinations would have raised overall accuracy to 81.05%.
  • Comparison with Cucerzan (2007). On the Microsoft/MSNBC news data the system scored 94.7% overall, against Cucerzan's 91.4%. The authors note the two numbers are not strictly comparable because the Wikipedia versions differ.

Interesting ideas

  • I1. NIL as a rankable option. Adding "no match" as a candidate inside the ranker lets features that signal absence be learned jointly with the match features. This could apply to any retrieval or linking task where abstaining is a valid answer, for example gazetteer matching for Ukrainian place names.
  • I2. Cheap, high-recall candidate generation before expensive ranking. Using indexable filters (acronyms, substrings, skip-bigram Dice) keeps the ranker tractable over millions of entries. Profiling showed that the costliest filter helped very few queries, a useful reminder to measure cost against recall gained for each component.
  • I3. KB-independent design with optional source-specific features. The core features avoid Wikipedia markup, and Wikipedia-only signals are added as a separate optional layer. This keeps the system portable to other curated knowledge bases, such as national registries or trade databases.
  • I4. Error analysis of missed candidates. Listing recall failures (inexact acronyms like PCF, metaphorical names like "Iron Lady", nicknames like "Queen City") shows which name-variation types still need alias resources or learned transliteration models. This is especially relevant for multilingual or transliterated names.
  • I5. Additive plus ablative feature studies, reported per class. Running both analyses and breaking accuracy down into NIL and non-NIL shows redundancy across feature groups and reveals trade-offs that a single overall accuracy number would hide.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition · Information extraction · Knowledge graphs & ontologies

Institutions

🎓Johns Hopkins UniversityBaltimore · United States151 works · 84 111 citations🎓University of Maryland, Baltimore CountyBaltimore · United States4 works · 819 citations

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{dredze2010entity,
  title = {{Entity Disambiguation for Knowledge Base Population}},
  author = {Dredze, Mark and McNamee, Paul and Rao, Delip and Gerber, Adam and Finin, Tim},
  booktitle = {Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)},
  year = {2010},
  doi = {10.13016/m29w0934d},
  url = {https://doi.org/10.13016/m29w0934d}
}

References

22 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Javier Artiles, Satoshi Sekine, and Julio Gonzalo. 2008. Web people search: results of the first evaluation and the plan for the second. In WWW. find
  2. Amit Bagga and Breck Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In Conference on Computational Linguistics (COLING). find our card
  3. Michele Banko and Oren Etzioni. 2008. The tradeoffs between open and traditional relation extraction. In Association for Computational Linguistics. find
  4. K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD Management of Data. find
  5. E. Boschee, R. Weischedel, and A. Zamanian. 2005. Automatic information extraction. In Conference on Intelligence Analysis. find
  6. Razvan C. Bunescu and Marius Pasca. 2006. Using encyclopedic knowledge for named entity disambiguation. In European Chapter of the Assocation for Computational Linguistics (EACL). find our card
  7. Peter Christen. 2006. A comparison of personal name matching: Techniques and practical issues. Technical Report TR-CS-06-02, Australian National University. find
  8. Silviu Cucerzan. 2007. Large-scale named entity disambiguation based on wikipedia data. In Empirical Methods in Natural Language Processing (EMNLP). find
  9. Markus Dreyer, Jason Smith, and Jason Eisner. 2008. Latent-variable modeling of string transductions with finite-state methods. In Empirical Methods in Natural Language Processing (EMNLP). find
  10. Anthony Fader, Stephen Soderland, and Oren Etzioni. 2009. Scaling Wikipedia-based named entity disambiguation to arbitrary web text. In WikiAI09 Workshop at IJCAI 2009. find
  11. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Knowledge Discovery and Data Mining (KDD). find
  12. Martin Klein and Michael L. Nelson. 2008. A comparison of techniques for estimating IDF values to generate lexical signatures for the web. In Workshop on Web Information and Data Management (WIDM). find
  13. Fangtao Li, Zhicheng Zhang, Fan Bu, Yang Tang, Xiaoyan Zhu, and Minlie Huang. 2009. THU QUANTA at TAC 2009 KBP and RTE track. In Text Analysis Conference (TAC). find
  14. Gideon S. Mann and David Yarowsky. 2003. Unsupervised personal name disambiguation. In Conference on Natural Language Learning (CONLL). find
  15. Andrew McCallum, Kamal Nigam, and Lyle Ungar. 2000. Efficient clustering of high-dimensional data sets with application to reference matching. In Knowledge Discovery and Data Mining (KDD). find
  16. Paul McNamee and Hoa Trang Dang. 2009. Overview of the TAC 2009 knowledge base population track. In Text Analysis Conference (TAC). find
  17. Paul McNamee, Mark Dredze, Adam Gerber, Nikesh Garera, Tim Finin, James Mayfield, Christine Piatko, Delip Rao, David Yarowsky, and Markus Dreyer. 2009. HLTCOE approaches to knowledge base population at TAC 2009. In Text Analysis Conference (TAC). find
  18. Marius Pasca. 2008. Turning web text and search queries into factual knowledge: hierarchical class attribute extraction. In National Conference on Artificial Intelligence (AAAI). find
  19. Massimo Poesio, David Day, Ron Artstein, Jason Duncan, Vladimir Eidelman, Claudio Giuliano, Rob Hall, Janet Hitzeman, Alan Jern, Mijail Kabadjov, Stanley Yong, Wai Keong, Gideon Mann, Alessandro Moschitti, Simone Ponzetto, Jason Smith, Josef Steinberger, Michael Strube, Jian Su, Yannick Versley, Xia… find
  20. Gerard Salton and Michael McGill. 1983. Introduction to Modern Information Retrieval. McGraw-Hill Book Company. find
  21. Erik Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Conference on Natural Language Learning (CONLL). find our card
  22. Zareen Syed, Tim Finin, and Anupam Joshi. 2008. Wikipedia as an ontology for describing documents. In Proceedings of the Second International Conference on Weblogs and Social Media. AAAI Press. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dredze-2010-entity-disambiguation-knowledge-base-population/