Elamur Research⌕
☰
home / papers / 2017 / Why we have switched from building full-fledged taxonomies to simply detecting hypernymy relations

Why we have switched from building full-fledged taxonomies to simply detecting hypernymy relations

Paper · 2017 · Jose Camacho-Collados

Summary

This short discussion paper asks why NLP research on is-a knowledge has moved away from inducing complete taxonomies and toward the narrower binary task of deciding whether two words stand in a hypernymy relation. The author gives two reasons. First, taxonomy evaluation has no standard benchmarks and depends on costly, hard-to-replicate manual judging of sampled edges. Second, supervised distributional and neural models fit naturally into a pairwise classification setup with ready-made datasets.

The paper presents no new system or experiments. Instead, it critiques current practice. Detection benchmarks are mostly derived from WordNet, Wikidata or DBPedia, so systems are rarely tested where those resources have gaps, such as specialized domains. Pairwise detection also does not directly produce the hypernyms of a given term, which is what applications like question answering actually need.

Two directions are proposed: richer, reproducible frameworks for evaluating taxonomies, and new hypernym discovery benchmarks scored with ranking metrics. The argument is relevant to anyone building or testing lexical hierarchies where gold resources are incomplete.

Key points

  • This is a single-author position paper (Sapienza University of Rome, 2017) with no experimental results. It surveys hypernymy identification as either pattern-based (rooted in Hearst's 1992 patterns) or distributional, and cites the LSTM model of Shwartz et al. (2016) that combines both signals.
  • The widely used BLESS dataset (Baroni and Lenci, 2011) also covers meronymy and co-hyponymy but includes only 200 distinct target concepts.
  • Espinosa-Anke et al. (2016a) evaluated a domain-aware hypernym discovery system both inside and outside Wikidata. The results suggest automatic systems cannot yet replace hand-built taxonomies in many domains.
  • Proposed hypernym discovery task: given a word such as "dog", the system returns hypernyms like mammal or animal. Evaluation would use MRR, MAP, R-Precision and P@k. Suggested resources include the SemEval taxonomy datasets (Bordea et al., 2015, 2016), the SemEval 2016 WordNet enrichment task, and the evaluation framework of Gupta et al. (2016), which also assesses granularity and generalization paths.
  • Following Boleda et al. (2017), the author recommends that benchmarks separate instantiation (Laika–dog) from concept-level hypernymy (dog–mammal).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Sapienza University of RomeRome · Italy64 works · 10,671 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Lexical semantics & word embeddings · 69 works

Cite

Cite as Jose Camacho-Collados. 2017. Why we have switched from building full-fledged taxonomies to simply detecting hypernymy relations. arXiv:1703.04178.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1703.04178 · arXiv 1703.04178

@misc{camachocollados2017why,
  title = {{Why we have switched from building full-fledged taxonomies to simply detecting hypernymy relations}},
  author = {Jose Camacho-Collados},
  year = {2017},
  doi = {10.48550/arxiv.1703.04178},
  eprint = {1703.04178},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1703.04178}
}

References

48 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Daniele Alfarone and Jesse Davis. 2015. Unsupervised learning of an is-a taxonomy from a limited domain-specific corpus. In Proceedings of IJCAI 2015. pages 1434–1441. find
  2. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In The semantic web, Springer, pages 722–735. find
  3. Marco Baroni, Raffaella Bernardi, Ngoc-Quynh Do, and Chung-chieh Shan. 2012. Entailment above the word level in distributional semantics. In Proceedings of EACL. pages 23–32.
  4. Marco Baroni and Alessandro Lenci. 2011. How we blessed distributional semantic evaluation. In Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics. Association for Computational Linguistics, pages 1–10.
  5. Jiang Bian, Yandong Liu, Eugene Agichtein, and Hongyuan Zha. 2008. Finding the right facts in the crowd: factoid question answering over social media. In Proceedings of the 17th international conference on World Wide Web. ACM, pages 467–476. find
  6. Guido Boella and Luigi Di Caro. 2013. Supervised learning of syntactic contexts for uncovering definitions and extracting hypernym relations in text databases. In Machine learning and knowledge discovery in databases, Springer, pages 64–79. find
  7. Gemma Boleda, Abhijeet Gupta, and Sebastian Padó. 2017. Instances and concepts in distributional space. In Proceedings of EACL (2). Association for Computational Linguistics, Valencia, Spain. find
  8. Georgeta Bordea, Paul Buitelaar, Stefano Faralli, and Roberto Navigli. 2015. Semeval-2015 task 17: Taxonomy extraction evaluation (texeval). In Proceedings of the SemEval workshop. find
  9. Georgeta Bordea, Els Lefever, and Paul Buitelaar. 2016. Semeval-2016 task 13: Taxonomy extraction evaluation (texeval-2). In SemEval-2016. Association for Computational Linguistics, pages 1081– 1091. find
  10. Jose Camacho-Collados and Roberto Navigli. 2017. BabelDomains: Large-Scale Domain Labeling of Lexical Resources. In Proceedings of EACL (2). Valencia, Spain. find
  11. Luis Espinosa-Anke, Jose Camacho-Collados, Claudio Delli Bovi, and Horacio Saggion. 2016a. Supervised distributional hypernym discovery via domain adaptation. In Proceedings of EMNLP. pages 424– 435. find
  12. Luis Espinosa-Anke, Horacio Saggion, Francesco Ronzano, and Roberto Navigli. 2016b. Extasem! extending, taxonomizing and semantifying domain terminologies. In Proceedings of AAAI. find
  13. Oren Etzioni, Michael Cafarella, Doug Downey, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S Weld, and Alexander Yates. 2005. Unsupervised named-entity extraction from the web: An experimental study. Artificial intelligence 165(1):91–134. find
  14. Stefano Faralli, Alexander Panchenko, Chris Biemann, and Simone P Ponzetto. 2016. Linked disambiguated distributional semantic networks. In International Semantic Web Conference. pages 56–64. find
  15. Tiziano Flati, Daniele Vannella, Tommaso Pasini, and Roberto Navigli. 2014. Two is bigger (and better) than one: the wikipedia bitaxonomy project. In Proceedings of ACL. find
  16. Tiziano Flati, Daniele Vannella, Tommaso Pasini, and Roberto Navigli. 2016. MultiWiBi: the Multilingual Wikipedia Bitaxonomy Project. Artificial Intelligence, to appear . find
  17. Trevor Fountain and Mirella Lapata. 2012. Taxonomy induction using hierarchical random graphs. In Proceedings of NAACL. Association for Computational Linguistics, pages 466–476. find
  18. Ruiji Fu, Jiang Guo, Bing Qin, Wanxiang Che, Haifeng Wang, and Ting Liu. 2014. Learning semantic hierarchies via word embeddings. In Proceedings of ACL.
  19. Amit Gupta, Francesco Piccinno, Mikhail Kozhevnikov, Marius Pasca, and Daniele Pighin. 2016. Revisiting taxonomy induction over wikipedia. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. Osaka, Japan, pages 2300–2309. http://aclweb.org… ↗
  20. Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the 14th conference on Computational linguistics. pages 539–545.
  21. Johannes Hoffart, Dragan Milchevski, and Gerhard Weikum. 2014. Stics: searching with strings, things, and cats. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. ACM, pages 1247–1248. find
  22. Eduard Hovy, Zornitsa Kozareva, and Ellen Riloff. 2009. Toward completeness in concept extraction and classification. In Proceedings of EMNLP. pages 948–957. find
  23. David Jurgens and Mohammad Taher Pilehvar. 2015. Reserating the awesometastic: An automatic extension of the wordnet taxonomy for novel terms. In HLT-NAACL. pages 1459–1465. find
  24. David Jurgens and Mohammad Taher Pilehvar. 2016. Semeval-2016 task 14: Semantic taxonomy enrichment. Proceedings of SemEval pages 1092–1102.
  25. Zornitsa Kozareva and Eduard Hovy. 2010. A semi-supervised method to learn and construct taxonomies using the web. In Proceedings of EMNLP. pages 1110–1118.
  26. Omer Levy, Steffen Remus, Chris Biemann, Ido Dagan, and Israel Ramat-Gan. 2015. Do supervised distributional methods really learn lexical inference relations? In Proceedings of NAACL 2015. Denver, Colorado, USA.
  27. Tuan Luu Anh, Jung-jae Kim, and See Kiong Ng. 2014. Taxonomy construction using syntactic contextual evidence. In Proceedings of EMNLP. pages 810– 819. find
  28. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM 38(11):39– 41.
  29. Ndapandula Nakashole, Gerhard Weikum, and Fabian M. Suchanek. 2012. PATTY: A Taxonomy of Relational Patterns with Semantic Types. In Proceedings of EMNLP-CoNLL. pages 1135–1145.
  30. Roberto Navigli and Paola Velardi. 2010. Learning word-class lattices for definition and hypernym extraction. In ACL. pages 1318–1327.
  31. Roberto Navigli, Paola Velardi, and Stefano Faralli. 2011. A graph-based algorithm for inducing lexical taxonomies from scratch. In IJCAI. pages 1872– 1877.
  32. Simone Paolo Ponzetto and Michael Strube. 2011. Taxonomy induction based on a collaboratively built knowledge repository. Artificial Intelligence 175(9-10):1737–1756. find
  33. John Prager, Jennifer Chu-Carroll, Eric W Brown, and Krzysztof Czuba. 2008. Question answering by predictive annotation. In Advances in Open Domain Question Answering, Springer, pages 307–347. find
  34. Stephen Roller and Katrin Erk. 2016. Relations such as hypernymy: Identifying and exploiting hearst patterns in distributional vectors for lexical entailment. In Proceedings of EMNLP. Austin, Texas, pages 2163–2172. https://aclweb.org/anthology/D16-1234. ↗
  35. Stephen Roller, Katrin Erk, and Gemma Boleda. 2014. Inclusive yet selective: Supervised distributional hypernymy detection. In Proceedings of COLING 2014. Dublin, Ireland.
  36. Enrico Santus, Alessandro Lenci, Qin Lu, and Sabine Schulte Im Walde. 2014. Chasing hypernyms in vector spaces with entropy. In EACL. pages 38–42. find
  37. Vered Shwartz, Yoav Goldberg, and Ido Dagan. 2016. Improving hypernymy detection with an integrated path-based and distributional method. In Proceedings of ACL. Berlin, Germany.
  38. Vered Shwartz, Enrico Santus, and Dominik Schlechtweg. 2017. Hypernyms under siege: Linguistically-motivated artillery for hypernymy detection. In Proceedings of EACL. Association for Computational Linguistics, Valencia, Spain.
  39. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. Advances in Neural Information Processing Systems 17 . find
  40. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2006. Semantic taxonomy induction from heterogenous evidence. In Proceedings of COLING/ACL 2006. pages 801–808. find
  41. Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: A core of semantic knowledge. In WWW. ACM, pages 697–706. find
  42. Paola Velardi, Stefano Faralli, and Roberto Navigli. 2013. OntoLearn Reloaded: A graph-based algorithm for taxonomy induction. Computational Linguistics 39(3):665–707.
  43. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM 57(10):78–85.
  44. Julie Weeds, Daoud Clarke, Jeremy Reffin, David Weir, and Bill Keller. 2014. Learning to distinguish hypernyms and co-hyponyms. In Proceedings of COL-ING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. pages 2249–2259.
  45. Mohamed Yahya, Klaus Berberich, Shady Elbassuoni, and Gerhard Weikum. 2013. Robust question answering over the web of linked data. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management. ACM, pages 1107–1116. find
  46. Ichiro Yamada, Jong-Hoon Oh, Chikara Hashimoto, Kentaro Torisawa, Jun’ichi Kazama, Stijn De Saeger, and Takuya Kawada. 2011. Extending wordnet with hypernyms and siblings acquired from wikipedia. In IJCNLP. pages 874–882. find
  47. Hui Yang and Jamie Callan. 2009. A metric-based framework for automatic taxonomy induction. In Proceedings of ACL/IJCNLP. Association for Computational Linguistics, pages 271–279. find
  48. Zheng Yu, Haixun Wang, Xuemin Lin, and Min Wang. 2015. Learning term embeddings for hypernymy identification. In Proceedings of IJCAI. pages 1390–1397. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/collados-2017-why-we-have-switched-building/