Elamur Research⌕
☰
home / papers / 2017 / Hypernyms under Siege: Linguistically-motivated Artillery for Hypernymy Detection

Hypernyms under Siege: Linguistically-motivated Artillery for Hypernymy Detection

Paper · 2017 · Vered Shwartz, Enrico Santus and Dominik Schlechtweg

Summary

The paper revisits unsupervised, distribution-based ways of telling hypernym pairs (e.g., dog → animal) apart from other semantic relations. The authors build many count-based distributional spaces from ukWaC combined with WaCkypedia EN. These spaces vary along two dimensions:

  • Context type: windows of size 2 or 5, directional or not; dependency neighbours; and a new joint parent–sister dependency context.
  • Weighting: raw frequency, PPMI, and PLMI.

On top of these spaces they score word pairs with a broad family of measures, grouped as similarity, inclusion, and informativeness measures. These include several SLQS variants that had not been tested before. Evaluation covers four benchmarks: BLESS, EVALution, Lenci/Benotto and Weeds.

No single combination of measure and settings wins everywhere. However, choosing the measure according to the linguistic properties of the competing relation yields stable results across datasets.

Supervised embedding-based classifiers score higher overall, but they mostly learn which words tend to be hypernyms, not the relation within a pair. When pairs were deliberately swapped, these classifiers broke down, while unsupervised scores held up. Because they are interpretable and need no training data, unsupervised measures remain a valuable, robust tool.

Key points

  • Corpus: ukWaC (about 2B words, crawled from .uk) plus WaCkypedia EN. The vocabulary is restricted to nouns, verbs and adjectives that occur at least 100 times.
  • Measures evaluated: cosine, Lin, APSyn, Weeds Precision, cosWeeds, ClarkeDE, APinc, balAPinc, invCL, SLQS, and the new variants SLQS Sub, SLQS Row and SLQS Row Sub.
  • Dataset sizes:
    • BLESS: 26,554 pairs
    • EVALution: 13,465 pairs
    • Lenci/Benotto: 5,010 pairs
    • Weeds: 2,928 pairs
  • Switched-pair test on BLESS: 139 mismatched pairs, such as (rifle, animal), were added to the validation set.
    • The best supervised model lost 42 points of average precision and labelled 121 of the 139 pairs as hypernyms.
    • The best unsupervised measure improved slightly.
  • Code and data: available at github.com/vered1986/UnsupervisedHypernymy. The joint dependency context proved highly discriminative.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Bar-Ilan UniversityRamat Gan · Israel146 works · 24,845 citations🎓University of StuttgartStuttgart · Germany94 works · 46,609 citations🎓Singapore University of Technology and DesignSingapore · Singapore54 works · 11,534 citations🎓Hong Kong Polytechnic UniversityHong Kong · China106 works · 5,190 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Lexical semantics & word embeddings · 65 works

Cite

Cite as Vered Shwartz, Enrico Santus and Dominik Schlechtweg. 2017. Hypernyms under Siege: Linguistically-motivated Artillery for Hypernymy Detection. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers.

The paper itself, not this page. BibTeX from the library's record of the work. arXiv 1612.04460

@inproceedings{shwartz2017hypernyms,
  title = {{Hypernyms under Siege: Linguistically-motivated Artillery for Hypernymy Detection}},
  author = {Vered Shwartz, Enrico Santus and Dominik Schlechtweg},
  booktitle = {Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers},
  year = {2017},
  doi = {10.18653/v1/e17-1007},
  eprint = {1612.04460},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/e17-1007}
}

References

39 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Zellig S. Harris. 1954. Distributional structure. Word, 10(2-3):146–162. find
  2. Marco Baroni and Alessandro Lenci. 2010. Distributional memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4):673–721.
  3. Marti A. Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In COLING 1992 Volume 2: The 15th International Conference on Computational Linguistics.
  4. Marco Baroni and Alessandro Lenci. 2011. Proceedings of the gems 2011 workshop on geometrical models of natural language semantics. In How we BLESSed distributional semantic evaluation, pages 1–10. Association for Computational Linguistics.
  5. Douwe Kiela, Laura Rimell, Ivan Vulić, and Stephen Clark. 2015. Exploiting image generality for lexical entailment detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume… find
  6. Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide web: a collection of very large linguistically processed web-crawled corpora. Language resources and evaluation, 43(3):209–226. find
  7. Lili Kotlerman, Ido Dagan, Idan Szpektor, and Maayan Zhitomirsky-Geffet. 2010. Directional distributional similarity for lexical inference. Natural Language Engineering, 16(04):359–389. find
  8. Marco Baroni, Raffaella Bernardi, Ngoc-Quynh Do, and Chung-chieh Shan. 2012. Entailment above the word level in distributional semantics. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 23–32. Association for Computational Linguis…
  9. Alessandro Lenci and Giulia Benotto. 2012. Identifying hypernyms in distributional semantic spaces. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International…
  10. Giulia Benotto. 2015. Distributional models for semantic relations: A sudy on hyponymy and antonymy. PhD Thesis, University of Pisa. find
  11. John A Bullinaria and Joseph P Levy. 2007. Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior research methods, 39(3):510–526. find
  12. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 302–308. Association for Computational Linguistics. find
  13. Emmanuele Chersoni, Enrico Santus, Alessandro Lenci, Philippe Blache, and Chu-Ren Huang. 2016. Representing verbs with rich contexts: an evaluation on verb similarity. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1967–1972. Association for Computat… find
  14. Omer Levy, Yoav Goldberg, and Ido Dagan. 2015a. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3:211– 225.
  15. Kenneth Ward Church and Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. Computational linguistics, 16(1):22–29.
  16. Omer Levy, Steffen Remus, Chris Biemann, and Ido Dagan. 2015b. Do supervised distributional methods really learn lexical inference relations? In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 970–9…
  17. Daoud Clarke. 2009. Context-theoretic semantics for natural language: an overview. In Proceedings of the Workshop on Geometrical Models of Natural Language Semantics, pages 112–119. Association for Computational Linguistics. find
  18. Ido Dagan, Dan Roth, and Mark Sammons. 2013. Recognizing textual entailment. Morgan & Claypool Publishers. find
  19. Dekang Lin. 1998. An information-theoretic definition of similarity. ICML, 98:296–304. find
  20. Stefan Evert. 2008. Corpora and collocations. Corpus linguistics. An international handbook, 2:223–233. find
  21. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119.
  22. Adriano Ferraresi. 2007. Building a very large corpus of english obtained by web crawling: ukwac. Masters thesis, University of Bologna, Italy. find
  23. Roberto Navigli, Paola Velardi, and Stefano Faralli. 2011. A graph-based algorithm for inducing lexical taxonomies from scratch. In Proceedings of Stefan Evert. 2005. The statistics of word cooccurrences: word pairs and collocations. Dissertation.
  24. Enrico Santus, Alessandro Lenci, Tin-Shing Chiu, Qin Lu, and Chu-Ren Huang. 2016b. Unsupervised measure of word similarity: How to outperform co-occurrence and vector cosine in vsms. In Thirtieth AAAI Conference on Artificial Intelligence, pages 4260–4261. the Twenty-Second International Joint Confe… find
  25. Sebastian Padó and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161–199.
  26. Vered Shwartz, Yoav Goldberg, and Ido Dagan. 2016. Improving hypernymy detection with an integrated path-based and distributional method. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2389–2398. Association for Computational…
  27. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Association for Computational Linguistics.
  28. Rion Snow, Daniel Jurafsky, and Andrew Y. Ng. 2005. Learning syntactic patterns for automatic hypernym discovery. In Advances in Neural Information Processing Systems 17, pages 1297–1304. MIT Press. find
  29. Laura Rimell. 2014. Distributional lexical entailment by topic coherence. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 511–519. Association for Computational Linguistics. find
  30. Rion Snow, Daniel Jurafsky, and Andrew Y. Ng. 2006. Semantic taxonomy induction from heterogenous evidence. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 801–808. Association for Comput… find
  31. Stephen Roller and Katrin Erk. 2016. Relations such as hypernymy: Identifying and exploiting hearst patterns in distributional vectors for lexical entailment. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2163–2172. Association for Computational Lin… find
  32. Peter D. Turney and Saif M. Mohammad. 2015. Experiments with three approaches to recognizing lexical entailment. Natural Language Engineering, 21(03):437–476. find
  33. Stephen Roller, Katrin Erk, and Gemma Boleda. 2014. Inclusive yet selective: Supervised distributional hypernymy detection. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1025– 1036. Dublin City University and Association for Co…
  34. Julie Weeds and David Weir. 2003. A general framework for distributional similarity. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 81–88.
  35. Julie Weeds, Daoud Clarke, Jeremy Reffin, David Weir, and Bill Keller. 2014. Learning to distinguish hypernyms and co-hyponyms. In Proceedings of COL-ING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 2249–2259. Dublin City University and Association fo…
  36. Gerard Salton and Michael J. McGill. 1986. Introduction to modern information retrieval. McGraw-Hill, Inc. find
  37. Enrico Santus, Alessandro Lenci, Qin Lu, and Sabine Schulte im Walde. 2014. Chasing hypernyms in vector spaces with entropy. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers, pages 38–42. Association for Computatio… find
  38. Enrico Santus, Frances Yung, Alessandro Lenci, and Chu-Ren Huang. 2015. Proceedings of the 4th workshop on linked data in linguistics: Resources and applications. In EVALution 1.0: an Evolving Semantic Dataset for Training and Evaluation of Distributional Semantic Models, pages 64–69. Association fo… find
  39. Enrico Santus, Alessandro Lenci, Tin-Shing Chiu, Qin Lu, and Chu-Ren Huang. 2016a. Nine features in a random forest to learn taxonomical semantic relations. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 4557–4564. European Language Resou… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shwartz-2017-hypernyms-under-siege-linguistically-motivated/