Elamur Research⌕
☰
home / papers / 2006 / Meaningful Clustering of Senses Helps Boost Word Sense Disambiguation Performance

Meaningful Clustering of Senses Helps Boost Word Sense Disambiguation Performance

Summary

Word Sense Disambiguation (WSD) systems built on WordNet have stalled near 70% accuracy, roughly the level at which human annotators agree with each other. The author argues that WordNet's very fine sense distinctions, not weak algorithms, are the main problem. To address this, the paper groups WordNet senses into coarser clusters by linking each one to an entry in the Oxford Dictionary of English (ODE), whose senses were organized into hierarchies by lexicographers.

Each sense in both dictionaries is represented as a bag of words drawn from its definition, its direct hypernyms and its domain labels. Two ways of matching WordNet senses to ODE senses are compared: a Lesk-style word-overlap score and a semantic method based on SSI (Structural Semantic Interconnections). WordNet senses that map to the same ODE entry form one cluster. The mapping is checked against manually built clusterings, and coarse-grained WSD is evaluated on the Senseval-3 English all-words task. With clustering, recall improves by a statistically significant margin, since related senses now support a shared cluster label.

Key points

  • Reported inter-annotator agreement with WordNet is 72.5% for the Senseval-3 all-words test set and 67.3% for Open Mind Word Expert. This puts a realistic upper bound for fine-grained WSD at about 70%.
  • On the same polysemous words, ODE averages 2.56 senses per word versus 3.21 in WordNet. For verbs the figures are 2.73 versus 3.75.
  • Sense descriptions are built as follows:
    • definitions are lemmatized and POS-tagged with the RASP parser;
    • WordNet domain labels come from Magnini and Cavaglià's subject field codes, with FACTOTUM excluded.
  • The lexical match counts shared lemmas and divides by the length of the shorter description. A threshold θ controls mapping, and WordNet senses that fall below it remain as singleton clusters.
  • Coarse-grained evaluation on Senseval-3 shows a significant recall gain (p < 0.05). The approach could also be applied to wordnets for other languages, given a suitably structured dictionary.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics Word sense disambiguation

 Institutions

🎓Sapienza University of RomeRome · Italy29 works · 6,807 citations

Cite

Cite as Roberto Navigli. 2006. Meaningful Clustering of Senses Helps Boost Word Sense Disambiguation Performance. Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1220175.1220189

@inproceedings{navigli2006meaningful,
  title = {{Meaningful Clustering of Senses Helps Boost Word Sense Disambiguation Performance}},
  author = {Roberto Navigli},
  booktitle = {Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics},
  year = {2006},
  doi = {10.3115/1220175.1220189},
  url = {https://doi.org/10.3115/1220175.1220189}
}

References

27 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre and Oier Lopez. 2003. Clustering wordnet word senses. In Proc. of Conf. on Recent Advances on Natural Language (RANLP). Borovets, Bulgary. find
  2. Ted Briscoe and John Carroll. 2002. Robust accurate statistical annotation of general text. In Proc. of 3 rd Conference on Language Resources and Evaluation. Las Palmas, Gran Canaria. our card
  3. Tim Chklovski and Rada Mihalcea. 2002. Building a sense tagged corpus with open mind word expert. In Proc. of ACL 2002 Workshop on WSD: Recent Successes and Future Directions. Philadelphia, PA. find
  4. Tim Chklovski and Rada Mihalcea. 2003. Exploiting agreement and disagreement of human annotators for word sense disambiguation. In Proc. of Recent Advances In NLP (RANLP 2003). Borovetz, Bulgaria. find
  5. Bart Decadt, Véronique Hoste, Walter Daelemans, and Antal van den Bosch. 2004. Gambl, genetic algorithm optimization of memory-based wsd. In Proc. of ACL/SIGLEX Senseval-3. Barcelona, Spain. find
  6. William B. Dolan. 1994. Word sense ambiguation: Clustering related senses. In Proc. of 15th Conference on Computational Linguistics (COLING). Morristown, N.J. find
  7. Philip Edmonds and Adam Kilgariff. 1998. Introduction to the special issue on evaluating word sense disambiguation systems. Journal of Natural Language Engineering, 8(4). find
  8. Christiane Fellbaum, Martha Palmer, Hoa Trang Dang, Lauren Delfs, and Susanne Wolf. 2001. Manual and automatic semantic annotation with wordnet. In Proc. of NAACL Workshop on WordNet and Other Lexical Resources. Pittsburgh, PA. find
  9. Christiane Fellbaum, editor. 1998. WordNet: an Electronic Lexical Database. MIT Press. find
  10. Michael Lesk. 1986. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine code from an ice cream cone. In Proc. of 5 th Conf. on Systems Documentation. ACM Press. find
  11. Bernardo Magnini and Gabriela Cavaglià. 2000. Integrating subject field codes into wordnet. In Proc. of the 2 nd Conference on Language Resources and Evaluation (LREC). Athens, Greece. our card
  12. Diana McCarthy. 2006. Relating wordnet senses for word sense disambiguation. In Proc. of ACL Workshop on Making Sense of Sense. Trento, Italy. find
  13. Rada Mihalcea and Ehsanul Faruque. 2004. Senselearner: Minimally supervised word sense disambiguation for all words in open text. In Proc. of ACL/SIGLEX Senseval-3. Barcelona, Spain. find
  14. Rada Mihalcea and Dan Moldovan. 2001. Automatic generation of a coarse grained wordnet. In Proc. of NAACL Workshop on WordNet and Other Lexical Resources. Pittsburgh, PA. find
  15. Jane Morris and Graeme Hirst. 1991. Lexical cohesion computed by thesaural relations as an indicator of the structure of text. Computational Linguistics, 17(1). our card
  16. Roberto Navigli and Paola Velardi. 2004. Learning domain ontologies from document warehouses and dedicated websites. Computational Linguistics, 30(2). find
  17. Roberto Navigli and Paola Velardi. 2005. Structural semantic interconnections: a knowledge-based approach to word sense disambiguation. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 27(7). find
  18. Hwee T. Ng, Chung Y. Lim, and Shou K. Foo. 1999. A case study on the inter-annotator agreement for word sense disambiguation. In Proc. of ACL Workshop: Standardizing Lexical Resources. College Park, Maryland. find
  19. Wim Peters, Ivonne Peters, and Piek Vossen. 1998. Automatic sense clustering in eurowordnet. In Proc. of the 1 st Conference on Language Resources and Evaluation (LREC). Granada, Spain. find
  20. Benjamin Snyder and Martha Palmer. 2004. The english all-words task. In Proc. of ACL 2004 SENSEVAL-3 Workshop. Barcelona, Spain. find
  21. Catherine Soanes and Angus Stevenson, editors. 2003. Oxford Dictionary of English. Oxford University Press. find
  22. Christopher Stokoe. 2005. Differentiating homonymy and polysemy in information retrieval. In Proc. of the Conference on Empirical Methods in Natural Language Processing. Vancouver, Canada. find
  23. Carlo Strapparava, Alfio Gliozzo, and Claudio Giuliano. 2004. Pattern abstraction and term similarity for word sense disambiguation. In Proc. of ACL/SIGLEX Senseval-3. Barcelona, Spain. find
  24. Noriko Tomuro. 2001. Tree-cut and a lexicon based on systematic polysemy. In Proc. of the Meeting of the NAACL. Pittsburgh, USA. find
  25. David Vickrey, Luke Biewald, Marc Teyssier, and Daphne Koller. 2005. Word sense disambiguation vs. statistical machine translation. In Proc. of Conference on Empirical Methods in Natural Language Processing. Vancouver, Canada. find
  26. Deniz Yuret. 2004. Some experiments with a naive bayes wsd system. In Proc. of ACL/SIGLEX Senseval-3. Barcelona, Spain. find
  27. Ying Zhao and George Karypis. 2004. Empirical and theoretical comparisons of selected criterion functions for document clustering. Machine Learning, 55(3). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/navigli-2006-meaningful-clustering-senses-helps-boost/