Elamur Research⌕
☰
home / papers / 2012 / Identifying hypernyms in distributional semantic spaces

Identifying hypernyms in distributional semantic spaces

Summary

Distributional Semantic Models capture how close two words are in meaning, but they do not show how the words are related. For example, dog is near both cat (a co-hyponym) and animal (a hypernym). Cosine is symmetric, so it cannot express that hypernymy runs in one direction only. The authors test whether asymmetric similarity measures, built on the Distributional Inclusion Hypothesis, can pick out hypernyms in a fully unsupervised setting.

Word vectors come from the TypeDM tensor. The test data is a BLESS subset in which each target noun is paired with coordinates, hypernyms, meronyms and random nouns. Three existing measures (WeedsPrec, cosWeeds, ClarkeDE) are compared against a Cosine baseline. The authors also introduce invCL, which rewards two things at once: the narrow term's features being contained in the broad term's, and the broad term having many features the narrow one lacks.

All directional measures move hypernyms above coordinates relative to Cosine. Only invCL, however, gives hypernyms a higher mean Average Precision than coordinates. This suggests that the contexts a hypernym does not share with its hyponym are a useful signal.

Key points

  • Features: Words are represented by TypeDM features (link + word pairs), weighted with Local Mutual Information (LMI); negative values are set to 0.

    • TypeDM covers 30,693 lemmas and 25,336 syntactic links and patterns.

  • Test set: The evaluation uses 14,547 BLESS tuples.

    • It covers 200 concrete English nouns from 17 classes.
    • It includes four relation types: COORD, HYPER, MERO and RANDOM-N.

  • New measure: invCL combines two quantities, both computed with ClarkeDE:

    • ClarkeDE(u,v), the inclusion of u in v;
    • one minus ClarkeDE(v,u), the non-inclusion of v in u.

  • Mean AP results (COORD / HYPER):

    MeasureCOORDHYPER
    Cosine0.790.23
    WeedsPrec0.450.40
    cosWeeds0.690.29
    ClarkeDE0.450.39
    invCL0.380.40

    invCL is the only measure that ranks HYPER above COORD.

  • Boxplot evaluation: Nearest-neighbour scores were z-normalized per concept and compared with a Tukey HSD test (α = 0.05).

    • All directional measures except cosWeeds significantly favour hypernyms over coordinates.
    • The authors attribute the cosWeeds result to its cosine component.

Interesting ideas

  • I1. Score what the broad term has extra. Inclusion alone cannot separate a hypernym from a co-hyponym, because two words with heavily overlapping feature sets each look "included" in the other. Adding a penalty for overlap in the reverse direction fixes this. The same trick could apply to any asymmetric relation, such as taxonomy induction for Ukrainian lexicons or broader/narrower categories in product or place hierarchies.
  • I2. Use co-hyponyms as the hard negative. Random pairs are easy to reject. Evaluating against coordinate terms reveals whether a method truly captures directionality. This is a good design for relation-classification benchmarks in general.
  • I3. Normalize scores per target item. Converting each concept's best-relatum scores to z-scores before pooling removes concept-specific variance. This makes measures with different scales comparable across many targets.
  • I4. Report two complementary metrics. Nearest-neighbour boxplots and per-concept Average Precision can disagree. Here, AP exposed that WeedsPrec and ClarkeDE still favour coordinates, even though the boxplots looked positive. Using both views guards against overly optimistic conclusions.
  • I5. Watch what mixing in symmetric similarity does. Combining an asymmetric score with cosine (as cosWeeds does) can dilute the directional signal. This is worth checking when building hybrid similarity scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics Lexical semantics & word embeddings

Cite

Cite as Alessandro Lenci and Giulia Benotto. 2012. Identifying hypernyms in distributional semantic spaces. *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{lenci2012identifying,
  title = {{Identifying hypernyms in distributional semantic spaces}},
  author = {Alessandro Lenci and Giulia Benotto},
  booktitle = {*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)},
  year = {2012},
  url = {https://aclanthology.org/S12-1012/}
}

References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marco Baroni and Alessandro Lenci. 2010. Distributional Memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4): 673–721. our card
  2. Marco Baroni and Alessandro Lenci. 2011. How we BLESSed distributional semantic evaluation. In Proceedings of the GEMS 2011 Workshop on Geometrical Models of Natural Language Semantics, EMNLP 2011, Edinburgh, Scotland, UK: 1–10. our card
  3. Daoud Clarke. 2009. Context-theoretic semantics for natural language: an overview. In Proceedings of the EACL 2009 Workshop on GEMS: GEometrical Models of Natural Language Semantics, Athens, Greece: 112– 119. find
  4. Katrin Erk. 2009a. Supporting inferences in semantic space: representing words as regions. In Proceedings of the 8th International Conference on Computational Semantics, Tilburg, January: 104–115. find
  5. Katrin Erk. 2009b. Representing words as regions in vector space. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL), Boulder, Colorado: 57–65. find
  6. Stefan Evert. 2005. The Statistics of Word Cooccurrences. Ph.D. dissertation, Stuttgart University. find
  7. Marti Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Proceedings of COLING 1992, Nantes, France: 539–545. find
  8. Lili Kotlerman, Ido Dagan, Idan Szpektor, and Maayan Zhitomirsky-Geffet. 2010. Directional distributional similarity for lexical inference. Natural Language Engineering, 16(04): 359–389. find
  9. Dekang Lin. 1998. Automatic Retrieval and Clustering of Similar Words. In Proceedings of the COLING-ACL 1998, Montreal, Canada: 768–774. our card
  10. Patrick Pantel and Marco Pennacchiotti. 2006. Espresso: Leveraging generic patterns for automatically harvesting semantic relations. In Proceedings of the COLING-ACL 2006, Sydney, Australia: 113–120. our card
  11. Idan Szpektor and Ido Dagan. 2008. Learning Entailment Rules for Unary Templates. In Proceedings of COLING 2008, Manchester, UK: 849–856. find
  12. Julie Weeds and David Weir. 2003. A general framework for distributional similarity. In Proceedings of the EMNLP 2003, Sapporo, Japan: 81–88. our card
  13. Julie Weeds, David Weir, and Diana McCarthy. 2004. Characterising measures of lexical distributional similarity. In Proceedings of COLING 2004, Geneva, Switzerland: 1015–1021. find
  14. Maayan Zhitomirsky-Geffet and Ido Dagan. 2005. The distributional inclusion hypotheses and lexical entailment. In Proceedings of ACL 2005, Ann Arbor, MI: 107–114. our card
  15. Maayan Zhitomirsky-Geffet and Ido Dagan. 2009. Bootstrapping distributional feature vector quality. Computational Linguistics, 35(3): 435-461. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lenci-2012-identifying-hypernyms-distributional-semantic-spaces/