Elamur Research⌕
☰
home / papers / 2016 / Word Sense Clustering and Clusterability

Word Sense Clustering and Clusterability

Paper · 2016 · Diana McCarthy, Marianna Apidianaki and Katrin Erk

Summary

Word sense disambiguation (WSD) and word sense induction (WSI) usually assume that a lemma's occurrences can be split cleanly into senses. In practice, some words have sharply separated meanings (crane), others are vague (thing), and many polysemous words fall in between. The authors define this property as partitionability and estimate it through clusterability, meaning how much cluster structure exists in a lemma's usage data. They represented instances using manual annotations: English paraphrases from LEXSUB and translations from CLLS. Two families of measures were tested. The first, intra-clust, uses k-means-based metrics from machine learning theory. The second, inter-clust, checks whether clusterings built from paraphrases and from translations agree. Gold estimates of partitionability came from the Usim graded similarity data set. Once polysemy is controlled for, clusterability generally tracks partitionability. The intra-clust metrics perform best and beat a soft-clustering overlap baseline. This matters because clusterability could indicate whether a word suits hard senses, soft clustering, or no clustering at all, and it could help plan annotation effort.

Key points

  • Gold standards from Usim: Uiaa is inter-annotator agreement. Umid is the share of mid-scale similarity judgments. Higher clusterability is expected to raise Uiaa and lower Umid.
  • Intra-clust measures: variance ratio (VR), worst pair ratio (WPR) and separability (SEP). The number of clusters k for these is supplied by a graph-based clustering. VR and SEP gave the most promising results.
  • Inter-clust measures: V and pF compare LEXSUB-based and CLLS-based clusterings. Their results were less consistent and did not always beat the baseline. The authors attribute this to their reliance on CLLS, which is generally weaker than LEXSUB.
  • Example correlations: on LEXSUB, low-polysemy band (29 lemmas), VR gave Spearman ρ ≈ −0.61 with Umid and ≈ 0.40 with Uiaa, both in the expected direction. Polysemy bands were analysed only if they contained at least five lemmas.
  • Future work: the authors plan to apply the measures to automatically generated paraphrases, translations and embeddings, and to compare the outputs of different clustering algorithms.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CambridgeCambridge · United Kingdom243 works · 121,396 citations🎓The University of Texas at AustinAustin · United States109 works · 56,942 citations🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🔬Laboratoire d'Informatique pour la Mécanique et les Sciences de l'IngénieurOrsay · France25 works · 7,462 citations🎓Université Paris-SaclayGif-sur-Yvette · France45 works · 6,127 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Word sense disambiguation · 83 worksLexical semantics & word embeddings · 63 works

Cite

Cite as Diana McCarthy, Marianna Apidianaki and Katrin Erk. 2016. Word Sense Clustering and Clusterability. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00247

@article{mccarthy2016word,
  title = {{Word Sense Clustering and Clusterability}},
  author = {Diana McCarthy, Marianna Apidianaki and Katrin Erk},
  journal = {Computational Linguistics},
  year = {2016},
  doi = {10.1162/coli\_a\_00247},
  url = {https://doi.org/10.1162/coli_a_00247}
}

References

50 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ackerman, Margareta and Shai Ben-David. 2009a. Clusterability: A theoretical study. Journal of Machine Learning Research -Proceedings Track, 5:1–8. find
  2. Ackerman, Margareta and Shai Ben-David. 2009b. Clusterability: A theoretical study. In Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS), pages 1–8, Clearwater Beach, FL. Agirre, Eneko and Philip Edmonds, editors. 2006. find
  3. Word Sense Disambiguation, Algorithms and Applications. Springer. Apidianaki, Marianna. 2008. Translation-oriented word sense induction based on parallel corpora. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08), pages 3269–3275, Marrakech. find
  4. Apidianaki, Marianna. 2009. Data-driven semantic analysis for multilingual WSD and lexical selection in translation. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL’09), pages 77–85, Athens. find
  5. Apidianaki, Marianna, Emilia Verzeni, and Diana McCarthy. 2014. Semantic clustering of pivot paraphrases. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 4270–4275, Reykjavik. Artiles, Javier, Enrique Amigó, and Julio Gonzalo. 2009. The rol… find
  6. Bannard, Colin and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 597–604, Ann Arbor, MI.
  7. Bansal, Mohit, John DeNero, and Dekang Lin. 2012. Unsupervised translation sense clustering. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL: HLT), pages 773–782, Montréal. find
  8. Biemann, Chris and Valerie Nygaard. 2010. Crowdsourcing WordNet. In Proceedings of the 5th Global WordNet Conference, Mumbai. find
  9. Carpuat, Marine and Dekai Wu. 2007. Improving statistical machine translation using word sense disambiguation. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 61–72, Prague.
  10. Chen, Jinying and Martha Palmer. 2009. Improving English verb sense disambiguation performance with linguistically motivated features and clear sense distinction boundaries. Journal of Language Resources and Evaluation, Special Issue on SemEval-2007, 43:181–208. find
  11. Copestake, Ann and Ted Briscoe. 1995. Semi-productive polysemy and sense extension. Journal of Semantics, 12:15–67. find
  12. Cruse, D. A. 2000. Aspects of the microstructure of word meanings. In Yael Ravin and Claudia Leacock, editors, Polysemy: Theoretical and Computational Approaches. Oxford University Press, pages 30–51. find
  13. Di Marco, Antonio and Roberto Navigli. 2013. Clustering and diversifying Web search results with graph-based word sense induction. Computational Linguistics, 39(3):709–754.
  14. Dyvik, Helge. 1998. Translations as semantic mirrors: From parallel corpus to Wordnet. In Proceedings of the Workshop Multilinguality in the Lexicon II at the 13th Biennial European Conference on Artificial Intelligence (ECAI’98), pages 24–44, Brighton. find
  15. Eom, Soojeong, Markus Dickinson, and Graham Katz. 2012. Using semi-experts to derive judgments on word sense alignment: A pilot study. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 605–611, Istanbul. find
  16. Epter, Scott, Mukkai Krishnamoorthy, and Mohammed Zaki. 1999. Clusterability detection and initial seed selection in large data sets. Technical Report 99-6, Rensselaer Polytechnic Institute, Computer Science Department, Troy, NY. find
  17. Erk, Katrin, Diana McCarthy, and Nick Gaylord. 2009. Investigations on word senses and word usages. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural… find
  18. Erk, Katrin, Diana McCarthy, and Nick Gaylord. 2013. Measuring word meaning in context. Computational Linguistics, 39(3):511–554. Fellbaum, Christiane, editor. 1998.
  19. WordNet, An Electronic Lexical Database. The MIT Press, Cambridge, MA. Goldberg, Mark K., Mykola Hayvanovych, and Malik Magdon-Ismail. 2010. Measuring similarity between sets of overlapping clusters. In SocialCom/PASSAT, pages 303–308, Minneapolis, MN. find
  20. Hanks, Patrick. 2000. Do word meanings exist? Computers and the Humanities. Senseval Special Issue, 34(1–2):205–215. find
  21. Hovy, Eduard, Mitch Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% solution. In Proceedings of the 2006 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL: HLT), pages 57–60, New York, NY… find
  22. Ide, Nancy, Tomaž Erjavec, and Dan Tufiş. 2002. Sense discrimination with parallel corpora. In Proceedings of the ACL’02 Workshop on Word Sense Disambiguation: Recent Successes and Future Directions, pages 54–60, Philadelphia, PA. find
  23. Ide, Nancy and Yorick Wilks. 2006. Making sense about sense. In Eneko Agirre and Phil Edmonds, editors, Word Sense Disambiguation, Algorithms and Applications. Springer, pages 47–73. find
  24. Jurgens, David and Ioannis Klapaftis. 2013. Semeval-2013 task 13: Word sense induction for graded and non-graded senses. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages… find
  25. Kilgarriff, Adam. 1998. ‘I don’t believe in word senses’. Computers and the Humanities, 31(2):91–113.
  26. Kilgarriff, Adam. 2006. Word Senses. In Eneko Agirre and Phil Edmonds, editors, Word Sense Disambiguation, Algorithms and Applications. Springer, pages 29–46. find
  27. Krippendorff, Klaus. 1980. Content Analysis: An Introduction to Methodology. find
  28. Sage Publications, Inc., Beverly Hills, CA. Lakoff, George. 1987. Women, Fire, and Dangerous Things: What Categories Reveal about the Mind. University of Chicago Press. find
  29. Landes, Shari, Claudia Leacock, and I. Tengi Randee. 1998. Building Semantic Concordances. In Christiane Fellbaum, editor, WordNet: an Electronic Lexical Database. MIT Press, Cambridge, MA, pages 199–237. find
  30. Lefever, Els and Veronique Hoste. 2010. SemEval-2007 Task 3: Cross-lingual word sense disambiguation. In Proceedings of the Fifth International Workshop on Semantic Evaluations (SemEval-2010), pages 15–20, Uppsala. find
  31. Manandhar, Suresh, Ioannis Klapaftis, Dmitriy Dligach, and Sameer Pradhan. 2010. SemEval-2010 Task 14: Word sense induction and disambiguation. In Proceedings of the Fifth International Workshop on Semantic Evaluation (SemEval-2010), pages 63–68, Uppsala. find
  32. McCarthy, Diana. 2009. Word sense disambiguation: An overview. Language and Linguistics Compass, 3(2):537–558. find
  33. McCarthy, Diana. 2011. Measuring similarity of word meaning in context with lexical substitutes and translations. In Alexander Gelbukh, editor, Computational Linguistics and Intelligent Text Processing, CICLing 2011, Pt. I (Lecture Notes in Computer Science, LNTCS 6608). Springer, pages 238–252. find
  34. McCarthy, Diana and Roberto Navigli. 2007. SemEval-2007 Task 10: English lexical substitution task. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 48–53, Prague.
  35. McCarthy, Diana and Roberto Navigli. 2009. The English lexical substitution task. Language Resources and Evaluation Special Issue on Computational Semantic Analysis of Language: SemEval-2007 and Beyond, 43(2):139–159. find
  36. Mihalcea, Rada, Ravi Sinha, and Diana McCarthy. 2010. SemEval-2010 Task 2: Cross-lingual lexical substitution. In Proceedings of the Fifth International Workshop on Semantic Evaluation (SemEval-2010), pages 9–14, Uppsala. find
  37. Mitchell, Jeff and Mirella Lapata. 2008. Vector-based models of semantic composition. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL’08: HLT), pages 236–244, Columbus, OH. find
  38. Navigli, Roberto. 2009. Word sense disambiguation: A survey. ACM Computing Surveys, 41(2):1–69. find
  39. Ostrovsky, Rafail, Yuval Rabani, Leonard J. Schulman, and Chaitanya Swamy. 2006. The effectiveness of Lloyd-type methods for the k-means problem. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pages 165–176, Berkeley, CA. find
  40. Palmer, Martha, Hoa Trang Dang, and Joseph Rosenzweig. 2000. Semantic tagging for the Penn treebank. In Proceedings of the Second International Conference on Language Resources and Evaluation (LREC’00), pages 699–704, Athens.
  41. Passonneau, Rebecca, Ansaf Salleb-Aouissi, Vikas Bhardwaj, and Nancy Ide. 2010. Word sense annotation of polysemous words by multiple annotators. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), pages 3244–3249, Valleta. find
  42. Passonneau, Rebecca J., Collin F. Baker, Christiane Fellbaum, and Nancy Ide. 2012. The MASC word sense corpus. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 3025–3030, Istanbul. find
  43. Pedersen, Ted. 2006. Unsupervised corpus-based methods for WSD. In Eneko Agirre and Phil Edmonds, editors, Word Sense Disambiguation, Algorithms and Applications. Springer, pages 131–166. find
  44. Resnik, Philip and David Yarowsky. 1997. A perspective on word sense disambiguation methods and their evaluation. In Proceedings of the ACL SIGLEX Workshop on Tagging Text with Lexical Semantics: Why, What, and How, pages 79–86, Washington, DC.
  45. Resnik, Philip and David Yarowsky. 2000. Distinguishing systems and distinguishing senses: New evaluation methods for word sense disambiguation. Natural Language Engineering, 5(3):113–133. find
  46. Rosenberg, Andrew and Julia Hirschberg. 2007. V-measure: A conditional entropy-based external cluster evaluation measure. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 410–420, Prague.…
  47. Sun, Lin and Anna Korhonen. 2009. Improving verb clustering with automatically acquired selectional preferences. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP’09), pages 638–647, Singapore. find
  48. Tuggy, David H. 1993. Ambiguity, Polysemy and Vagueness. Cognitive Linguistics, 4(2):273–290. Véronis, Jean. 2004. HyperLex: Lexical Cartography for Information Retrieval. Computer Speech & Language, 18(3):223–252. find
  49. Yuret, Deniz. 2007. KU: Word sense disambiguation by substitution. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 207–214, Prague. find
  50. Zhang, Bin. 2001. Dependence of clustering algorithm performance on clustered-ness of data. Technical report HP-2001-91, Hewlett-Packard Labs. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mccarthy-2016-word-sense-clustering-clusterability/