Elamur Research
☰
home / papers / 2016 / Improving Hypernymy Detection with an Integrated Path-based and Distributional Method

Improving Hypernymy Detection with an Integrated Path-based and Distributional Method

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P16-1226.pdf

Summary

The paper tackles automatic hypernymy detection: deciding whether term y is a more general category of term x (for example, whether bird is a hypernym of parrot). Two families of methods existed. Distributional methods compare the contexts of each term separately. Path-based methods look at the dependency paths linking the two terms when they co-occur. Path-based methods had fallen behind because treating each path as a separate feature is very sparse. The authors propose HypeNET. It encodes each dependency path edge by edge with an LSTM, then averages the path vectors of a term-pair and feeds them to a softmax classifier. Training data comes from term-pairs extracted from knowledge resources. This path-only model already rivals supervised distributional methods. When word embeddings of x and y are added to the same network, the combined model clearly beats the previous state of the art. The work shows that the two signals complement each other. It also shows that neural path encoding can generalize across paths that are semantically, not just syntactically, similar.

Key points

  • Edge representation: each path edge concatenates embeddings of four parts: the lemma, POS tag, dependency label and direction (lemma/POS/dep/dir). Following Snow et al. (2004), "satellite edges" are added so that patterns like such Y as X can be captured.
  • Integration gains: the integrated model improves by up to 14 F1 points over either the path-based or the distributional model alone.
  • Path learning compared with baselines:
    • Snow's baseline learns rigid, verbatim paths.
    • Its generalized variant over-abstracts, e.g. to X VERB Y from.
    • HypeNET learns finer variants such as X become/remain Y from.
    • On the lexical split, however, HypeNET mostly falls back to the noisy X is Y path.
  • False positives: about 21% are synonyms, about 29% are reversed hyponym pairs (often near-synonyms), and about 21% are hypernymy-like relations labeled negative in the resources.
  • False negatives: in a sample of 50, 80% involved few co-occurrences in the corpus. Code and data are released at github.com/vered1986/HypeNET.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

109 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 108 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-1226 · arXiv 1603.06076

@inproceedings{shwartz2016improving,
  title = {{Improving Hypernymy Detection with an Integrated Path-based and Distributional Method}},
  author = {Shwartz, Vered and Goldberg, Yoav and Dagan, Ido},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1226},
  eprint = {1603.06076},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p16-1226}
}

References

31 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. We would like to thank Omer Levy for his involvement and assistance in the early stage of this project and Enrico Santus for helping us by computing the results of SLQS (Santus et al., 2014) on our dataset. This work was partially supported by an Intel ICRI-CI grant, the Israel Science Foundation gr… find
  2. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. Springer. find
  3. Marco Baroni and Alessandro Lenci. 2011. How we blessed distributional semantic evaluation. In Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, pages 1–10. find
  4. Marco Baroni, Raffaella Bernardi, Ngoc-Quynh Do, and Chung-chieh Shan. 2012. Entailment above the word level in distributional semantics. In EACL, pages 23–32. find
  5. Claudia Borg, Mike Rosner, and Gordon Pace. 2009. Evolutionary algorithms for definition extraction. In Proceedings of the 1st Workshop on Definition Extraction, pages 26–32. find
  6. Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka Jr, and Tom M Mitchell. 2010. Toward an architecture for never-ending language learning. In AAAI. find
  7. Christiane Fellbaum. 1998. WordNet. Wiley Online Library. find
  8. Katrin Fundel, Robert Küffner, and Ralf Zimmer. 2007. Relexrelation extraction using dependency parse trees. Bioinformatics, pages 365–371. find
  9. Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In ACL, pages 539– 545. find
  10. Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2009. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In SemEval, pages 94–99. find our card
  11. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, pages 1735–1780. find
  12. Nobuhiro Kaji and Masaru Kitsuregawa. 2008. Using hidden markov random fields to combine distributional and pattern-based word clustering. In COL-ING, pages 401–408. find
  13. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  14. Lili Kotlerman, Ido Dagan, Idan Szpektor, and Maayan Zhitomirsky-Geffet. 2010. Directional distributional similarity for lexical inference. NLE, pages 359–389. find
  15. Zornitsa Kozareva and Eduard Hovy. 2010. A semi-supervised method to learn and construct taxonomies using the web. In EMNLP, pages 1110– 1118. find
  16. Omer Levy, Steffen Remus, Chris Biemann, and Ido Dagan. 2015. Do supervised distributional methods really learn lexical inference relations. NAACL. find our card
  17. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. In NIPS. find
  18. Dekang Lin. 1998. An information-theoretic definition of similarity. In ICML, pages 296–304. find
  19. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2006. Semantic taxonomy induction from heterogenous evidence. In ACL, pages 801–808. find
  20. Yang Liu, Furu Wei, Sujian Li, Heng Ji, Ming Zhou, and Houfeng Wang. 2015. A dependency-based neural network for relation classification. arXiv preprint arXiv:1507.04646. find
  21. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111–3119. find
  22. Shachar Mirkin, Ido Dagan, and Maayan Geffet. 2006. Integrating pattern-based and distributional similarity methods for lexical entailment acquisition. In COLING and ACL, pages 579–586. find
  23. Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In WWW, pages 697–706. find
  24. Peter D Turney. 2006. Similarity of semantic relations. CL, pages 379–416. find
  25. Denny Vrandečić. 2012. Wikidata: A new platform for collaborative data collection. In WWW, pages 1063–1064. find
  26. Julie Weeds and David Weir. 2003. A general framework for distributional similarity. In EMLP, pages 81–88. find
  27. Ndapandula Nakashole, Gerhard Weikum, and Fabian Suchanek. 2012. Patty: a taxonomy of relational patterns with semantic types. In EMNLP and CoNLL, pages 1135–1145. find
  28. Julie Weeds, Daoud Clarke, Jeremy Reffin, David Weir, and Bill Keller. 2014. Learning to distinguish hypernyms and co-hyponyms. In COLING, pages 2249–2259. find
  29. Roberto Navigli and Paola Velardi. 2010. Learning word-class lattices for definition and hypernym extraction. In ACL, pages 1318–1327. find
  30. Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. 2015. Classifying relations via long short term memory networks along shortest dependency paths. In EMNLP. find
  31. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532–1543. find