Elamur Research⌕
☰
home / papers / 2014 / Learning to Distinguish Hypernyms and Co-Hyponyms

Learning to Distinguish Hypernyms and Co-Hyponyms

Summary

Distributional thesauri mix many relations among a word's nearest neighbours, including hypernyms, co-hyponyms and other related words. Common similarity measures are symmetric, so they cannot show which word in a pair such as cat/animal is the more general one. That direction matters for lexical entailment. The authors frame the task as classifying a pair of similar nouns into one of three cases: one entails the other, the reverse, or they are co-hyponyms. They build dependency-based noun vectors from a 2011 Wikipedia dump parsed with the Malt Parser. A linear SVM is then trained on pair representations made by combining the two words' vectors in different ways. These are compared with unsupervised directional measures based on distributional generality (balAPinc, invCL and others) and with kNN. The evaluation uses BLESS plus new, larger data sets derived from WordNet, built so that test words never appear in training. On the WordNet data the SVMs clearly beat the unsupervised measures, cutting error by over 15%. The best vector operation depends on which relation is targeted.

Key points

  • Data and vectors:
    • Source is a June 2011 Wikipedia dump, with text extracted by wp2txt.
    • The text was POS-tagged, lemmatised and dependency-parsed with the Malt Parser.
    • Only nouns occurring at least 100 times were kept.
  • Pair representations:
    • The SVM variants are svmDIFF, svmCAT, svmADD, svmMULT and svmSING. They use difference, concatenation, sum, element-wise product, or one word's vector alone.
    • knnDIFF serves as a non-linear comparison.
  • Main result:
    • Supervised SVMs beat unsupervised measures at the 0.01% significance level, an absolute error-rate reduction above 15%.
    • On the hyponym WN set, svmDIFF outperforms knnDIFF by 19–25%.
  • Operation choice:
    • For entailment, DIFF performs as well as CAT.
    • For the co-hyponym (coordinate) task, ADD beats CAT by 4%.
    • On the co-hyponym WN set, the unsupervised measures fail entirely.
  • Data sets and code:
    • On BLESS the SVMs did poorly, because excluding test words from training leaves too little data.
    • The WordNet-derived data sets are released as a benchmark, and the code is open-sourced.

Interesting ideas

  • I1. Match the vector combination to the relation. Subtracting vectors suits asymmetric relations like entailment, while adding them suits symmetric ones like co-hyponymy. This is a cheap design rule for any classifier that operates on word pairs.
  • I2. Test generalisation with a lexical split. Forbid any test word, and the words it relates to, from appearing in training. This stops the model from memorising which words tend to be general, a check that is worth copying for any pair-classification benchmark.
  • I3. Use a single-word baseline (svmSING). If a classifier that sees only one word already does well, the data set leaks information through word-level properties. This is a useful sanity check for relation data sets.
  • I4. Compare a linear SVM against kNN on the same features. The gap between them shows that positive and negative pairs sit close together yet are linearly separable. Such diagnostics help explain why a model works, not just whether it does.
  • I5. Look past corpus size when choosing an evaluation set. A small resource like BLESS can make supervised methods look weak only because a fair split leaves little training data. Deriving a larger set from WordNet fixed this, and the same move could help low-resource languages that have a wordnet.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

91 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 87 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{weeds2014learning,
  title = {{Learning to Distinguish Hypernyms and Co-Hyponyms}},
  author = {Weeds, Julie and Clarke, Daoud and Reffin, Jeremy and Weir, David and Keller, Bill},
  booktitle = {Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers},
  year = {2014},
  url = {https://aclanthology.org/C14-1212/}
}

References

39 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marco Baroni and Alessandro Lenci. 2011. How we BLESSed distributional semantic evaluation. In Proceedings of the GEMS 2011 workshop on Geometric Models of Natural Language Semantics, EMNLP 2011. find our card
  2. Marco Baroni and Roberto Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. find
  3. Marco Baroni, Raffaella Bernardi, Ngoc-Quynh Do, and Chung-chieh Shan. 2012. Entailment above the word level in distributional semantics. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 23–32. Association for Computational Linguis… find our card
  4. Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2010. Global learning of focused entailment graphs. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1220–1229, Uppsala, Sweden, July. Association for Computational Linguistics. find
  5. Shane Bergsma, Aditya Bhargava, Hua He, and Grzegorz Kondrak. 2010. Predicting the semantic compositionality of prefix verbs. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 293–303, Cambridge, MA, October. Association for Computational Linguistics. find
  6. Danushka Bollegala, David Weir, and John Carroll. 2011. Using multiple sources to construct a sentiment sensitive thesaurus for cross-domain sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT 2011… find
  7. Kenneth Ward Church and Patrick Hanks. 1989. Word association norms, mutual information, and lexicography. In Proceedings of the 27th Annual Meeting on Association for Computational Linguistics, ACL ’89, pages 76–83, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  8. Daoud Clarke. 2009. Context-theoretic semantics for natural language: an overview. In Proceedings of the Workshop of Geometric Models for Natural Language Semantics. find
  9. James Curran. 2004. From Distributional to Semantic Similarity. Ph.D. thesis, University of Edinburgh. find
  10. Georgiana Dinu and Stefan Thater. 2012. Saarland: Vector-based models of semantic textual similarity. In Proceedings of the First Joint Conference on Lexical and Computational Semantics. find
  11. Christaine Fellbaum, editor. 1989. WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, Massachusetts. find
  12. Trevor Fountain and Mirella Lapata. 2012. Taxonomy induction using hierarchical random graphs. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 466–476, Montréal, Canada, June. find
  13. Maayan Geffet and Ido Dagan. 2005. Lexical entailment and the distributional inclusion hypothesis. In Proceedings of the 43rd meeting of the Association for Computational Liuguistics (ACL), pages 107–114. find
  14. Edward Grefenstette, Mehrnoosh Sadrzadeh, Stephen Clark, Bob Coecke, and Stephen Pulman. 2011. Concrete sentence spaces for compositional distributional models of meaning. Proceedings of the 9th International Conference on Computational Semantics (IWCS 2011), pages 125–134. find
  15. Zelig Harris. 1954. Distributional structure. Word, 10:146–162. find
  16. Mitesh Khapra, Anup Kulkarni, Saurabh Sohoney, and Pushpak Bhattacharyya. 2010. All words domain adapted WSD: Finding a middle ground between supervision and unsupervision. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1532–1541, Uppsala, Sweden, J… find
  17. Adam Kilgarriff and Colin Yallop. 2000. What’s in a thesaurus? Conference on Language Resources and Evaluation (LREC2000). find
  18. Lili Kotlerman, Ido Dagan, Idan Szpektor, and Maayan Zhitomirsky-Geffet. 2010. Directional distributional similarity for lexical inference. Special Issue of Natural Language Engineering on Distributional Lexical Semantics, 4(16):359–389. find
  19. Lillian Lee. 1999. Measures of distributional similarity. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 25–32, College Park, Maryland, USA, June. find
  20. Alessandro Lenci and Giulia Benotto. 2012. Identifying hypernyms in distributional semantic spaces. In Proceedings of the First Joint Conference on Lexical and Computational Semantics (*Sem). find
  21. Dekang Lin. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the 17th International Conference on Computational Linguistics (COLING 1998). find our card
  22. Tara McIntosh. 2010. Unsupervised discovery of negative categories in lexicon bootstrapping. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 356–365, Cambridge, MA, October. Association for Computational Linguistics. find
  23. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751, Atlanta, Georgia,… find our card
  24. Tristan Miller, Chris Biemann, Torsten Zesch, and Iryna Gurevych. 2012. Using distributional similarity for lexical expansion in knowledge-based word sense disambiguation. In Proceedings of COLING 2012, pages 1781–1796, Mumbai, India, December. find
  25. Jeff Mitchell and Mirella Lapata. 2008. Vector-based models of semantic composition. In Proceedings of ACL-08: HLT, pages 236–244, Columbus, Ohio, June. Association for Computational Linguistics. find
  26. Joakim Nivre. 2004. Incrementality in deterministic dependency parsing. In Proceedings of the ACL workshop on Incremental Parsing, pages 50–57. find our card
  27. In Proceedings of the 2nd International Marek Rei and Ted Briscoe. 2013. Parser lexicalisation through self-learning. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 391–400, Atlanta, Georgia, Ju… find
  28. Enrico Santus, Alessandro Lenci, Qin Lu, and Sabine Schulte Im Walde. 2014. Chasing hypernyms in vector spaces with entropy. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 38–42, Gothenburg, Sweden, April. find
  29. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. Advances in Neural Information Processing Systems 17. find
  30. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2006. Semantic taxonomy induction from heterogenous evidence. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 801–808. Association for Com… find
  31. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201–1211. find our card
  32. György Szarvas, Chris Biemann, and Iryna Gurevych. 2013. Supervised all-words lexical substitution using delexicalized features. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1131–1141, Atlant… find
  33. Idan Szpektor and Ido Dagan. 2008. Learning entailment rules for unary templates. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 849–856, Manchester, UK, August. find
  34. Julie Weeds and David Weir. 2003. A general framework for distributional similarity. In Michael Collins and Mark Steedman, editors, Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 81–88. find
  35. Julie Weeds, David Weir, and Diana McCarthy. 2004. Characterising measures of lexical distributional similarity. In Proceedings of Coling 2004, pages 1015–1021, Geneva, Switzerland, Aug 23–Aug 27. COLING. find
  36. Julie Weeds, David Weir, and Jeremy Reffin. 2014. Distributional composition using higher-order dependency vectors. In Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality, EACL 2014, Gothenburg, Sweden, April. find
  37. Dominic Widdows. 2008. Semantic vector products: Some initial investigations. In Proceedings of the Second Symposium on Quantum Interaction, Oxford, UK, pages 1–8. find
  38. Wen-tau Yih, Geoffrey Zweig, and John Platt. 2012. Polarity inducing latent semantic analysis. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1212–1222, Jeju Island, Korea, July. Association for Compu… find
  39. Chen Zhang and Joyce Chai. 2010. Towards conversation entailment: An empirical investigation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 756–766, Cambridge, MA, October. Association for Computational Linguistics. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/weeds-2014-learning-distinguish-hypernyms-co-hyponyms/