Elamur Research⌕
☰
home / papers / 2018 / Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora

Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora

Paper · 2018 · Stephen Roller, Douwe Kiela and Maximilian Nickel

Summary

The paper asks whether classic Hearst-style lexico-syntactic patterns, often dismissed as too sparse, can still compete with distributional methods based on the Distributional Inclusion Hypothesis (DIH) for unsupervised hypernym detection. The authors mined is-a pairs from Gigaword and Wikipedia using a small set of hand-written patterns, with preprocessing done in CoreNLP 3.8.0. They scored candidate pairs in four ways: raw extraction probability, PPMI, and low-rank versions of both obtained via truncated SVD. The SVD variants infer missing pairs from similarities between words. These models were compared against Cosine, WeedsPrec, invCL and SLQS, all computed on a large dependency-based space built from UkWaC and Wikipedia. Across nine benchmarks covering detection, direction prediction and graded entailment, the pattern models won almost everywhere. The SVD-smoothed PPMI variant (spmi) was usually the strongest. The work matters because it shows that the tightly constrained contexts captured by patterns carry hypernymy signal that DIH measures miss, and that matrix factorization largely solves the sparsity problem.

Key points

  • The pattern corpus has roughly 4.5M pair matches in total, which reduce to 431K distinct pairs involving 243K distinct terms. The distributional baseline space covers 218K words over 732K context dimensions.
  • Evaluation used three task types. Detection was measured with Average Precision on BLESS, EVAL, LEDS, SHWARTZ and WBLESS. Direction was measured with accuracy on BLESS, WBLESS and BIBLESS. Graded entailment was measured with Spearman ρ on HYPERLEX nouns.
  • Gains in detection were large: spmi reached 0.76 AP on BLESS versus 0.19 for the best distributional measure, and 0.96 versus 0.69 on WBLESS.
  • In direction prediction on BLESS, the SVD models reached 0.96 accuracy, surpassing even the supervised state-of-the-art model of Vulić and Mrkšić (2017). On HYPERLEX, the plain p(x, y) model achieved the best score (ρ = 0.62).
  • Sparse pattern models struggled on SHWARTZ, which has a long tail of rare words, and on EVAL, which contains many verbs and adjectives that the patterns do not cover. Code is released as facebookresearch/hypernymysuite.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MetaMenlo Park · United States214 works · 241,492 citations🏢Meta (Israel)Tel Aviv · Israel46 works · 42,810 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksBiochemistry, Genetics and Molecular Biology · 263 worksMolecular Biology · 260 works

🧭 Topics 1Research subjects and languages the work is about.Lexical semantics & word embeddings · 65 works

Cite

Cite as Stephen Roller, Douwe Kiela and Maximilian Nickel. 2018. Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-2057 · arXiv 1806.03191

@inproceedings{roller2018hearst,
  title = {{Hearst Patterns Revisited: Automatic Hypernym Detection from Large Text Corpora}},
  author = {Stephen Roller, Douwe Kiela and Maximilian Nickel},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-2057},
  eprint = {1806.03191},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p18-2057}
}

References

23 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marco Baroni, Raffaella Bernardi, Ngoc-Quynh Do, and Chung-chieh Shan. 2012. Entailment above the word level in distributional semantics. In Proceedings of the 2012 Conference of the European Chapter of the Association for Computational Linguists, pages 23–32, Avignon, France.
  2. Scott Cederberg and Dominic Widdows. 2003. Using lsa and noun coordination information to improve the recall and precision of automatic hyponymy extraction. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003. find
  3. Daoud Clarke. 2009. Context-theoretic semantics for natural language: An overview. In Proceedings of the 2011 Workshop on GEometrical Models of Natural Language Semantics, pages 112–119, Athens, Greece. Association for Computational Linguistics. find
  4. Marti A. Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the 1992 Conference on Computational Linguistics, pages 539–545, Nantes, France.
  5. Douwe Kiela, Laura Rimell, Ivan Vulić, and Stephen Clark. 2015. Exploiting image generality for lexical entailment detection. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume… find
  6. Lili Kotlerman, Ido Dagan, Idan Szpektor, and Maayan Zhitomirsky-Geffet. 2010. Directional distributional similarity for lexical inference. Natural Language Engineering, 16:359–389. find
  7. Alessandro Lenci and Giulia Benotto. 2012. Identifying hypernyms in distributional semantic spaces. In Proceedings of the First Joint Conference on Lexical and Computational Semantics, pages 75–79, Montréal, Canada. Association for Computational Linguistics.
  8. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 302–308, Baltimore, Maryland. Association for Computational Linguistics. find
  9. Omer Levy, Steffen Remus, Chris Biemann, and Ido Dagan. 2015. Do supervised distributional methods really learn lexical inference relations? In Proceedings of the 2015 North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 970–976, Denver, Colorad…
  10. Kim Anh Nguyen, Maximilian Keper, Sabine Schulte im Walde, and Ngoc Thang Vu. 2017. Hierarchical Embeddings for Hypernymy Detection and Directionality. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 233–243, Copenhagen, Denmark. find
  11. Maximillian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representations. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 6338–6347. Curran Associates,…
  12. Sebastian Padó and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161–199.
  13. Stephen Roller and Katrin Erk. 2016. Relations such as hypernymy: Identifying and exploiting hearst patterns in distributional vectors for lexical entailment. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, USA. Association for Computational… find
  14. Enrico Santus, Alessandro Lenci, Qin Lu, and Sabine Schulte im Walde. 2014. Chasing hypernyms in vector spaces with entropy. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers, pages 38–42, Gothenburg, Sweden. Associ… find
  15. Enrico Santus, Frances Yung, Alessandro Lenci, and Chu-Ren Huang. 2015. EVALution 1.0: An evolving semantic dataset for training and evaluation of distributional semantic models. In Proceedings of the Fourth Workshop on Linked Data in Linguistics: Resources and Applications, pages 64–69, Beijing, Ch… find
  16. Vered Shwartz, Yoav Goldberg, and Ido Dagan. 2016. Improving hypernymy detection with an integrated path-based and distributional method. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2389– 2398, Berlin, Germany. Association for Computational Lingu…
  17. Vered Shwartz, Enrico Santus, and Dominik Schlechtweg. 2017. Hypernyms under siege: Linguistically-motivated artillery for hypernymy detection. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 65–75, Valencia…
  18. Rion Snow, Daniel Jurafsky, and Andrew Y. Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, pages 1297–1304. MIT Press. find
  19. Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, and Anna Korhonen. 2017. Hyperlex: A large-scale evaluation of graded lexical entailment. Computational Linguistics, 43(4):781–835.
  20. Ivan Vulić and Nikola Mrkšić. 2017. Specialising word vectors for lexical entailment. CoRR, abs/1710.06371.
  21. Julie Weeds, Daoud Clarke, Jeremy Reffin, David Weir, and Bill Keller. 2014. Learning to distinguish hypernyms and co-hyponyms. In Proceedings of the 2014 International Conference on Computational Linguistics, pages 2249–2259, Dublin, Ireland.
  22. Julie Weeds, David Weir, and Diana McCarthy. 2004. Characterising measures of lexical distributional similarity. In Proceedings of the 2004 International Conference on Computational Linguistics, pages 1015–1021, Geneva, Switzerland. find
  23. Maayan Zhitomirsky-Geffet and Ido Dagan. 2005. The distributional inclusion hypotheses and lexical entailment. In Proceedings of the 2005 Annual Meeting of the Association for Computational Linguistics, pages 107–114, Ann Arbor, Michigan.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/roller-2018-hearst-patterns-revisited-automatic-hypernym/