Elamur Research
☰
home / papers / 1995 / Using Information Content to Evaluate Semantic Similarity in a Taxonomy

Using Information Content to Evaluate Semantic Similarity in a Taxonomy

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "All rights reserved — free to read at the source" does not allow redistributing the text itself. Original — ijcai.org/Proceedings/95-1/Papers/059.pdf

Summary

The paper addresses how to estimate semantic similarity between concepts in an IS-A taxonomy. The standard edge-counting approach treats every link as an equal distance, but real hierarchies such as WordNet vary widely in how much meaning a single link spans. Resnik's alternative attaches corpus-derived probabilities to taxonomy nodes. A concept's information content is defined as its negative log probability. Two concepts are then scored by the information content of the most informative concept that subsumes both. For words, the maximum is taken over all sense pairs. Concept frequencies come from noun counts in the Brown Corpus, propagated upward through the WordNet noun hierarchy. Evaluation compares system scores with the Miller and Charles human ratings. The information-content measure clearly outperforms edge counting and comes reasonably close to inter-human agreement. The work matters for two reasons. It gives a simple measure that is robust to uneven link density. It also shows that a fixed knowledge resource can be adapted to different domains through corpus statistics.

Key points

  • Resources: WordNet's noun taxonomy (about 50,000 nodes) and the Brown Corpus (about 1 million words). Each noun occurrence counts toward every class that contains it, and probabilities are relative frequencies.
  • Benchmark: Miller and Charles (1991) collected ratings for 30 noun pairs from 38 subjects on a 0–4 scale. Resnik's replication with 10 subjects correlated at .96 with the original means.
  • Human upper bound: the average per-subject correlation was r = 0.8848 over 30 pairs and r = 0.9015 over the 28 pairs scorable in WordNet (one noun is missing).
  • Main result: information content reached r = 0.79, against r = 0.66 for edge counting. A plain-probability variant was also compared.
  • Weakness: maximizing over senses can produce spurious matches. For example, horse scores as similar to tobacco through a slang sense meaning heroin, which places both words under narcotics.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Cited by

226 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 221 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. arXiv cmp-lg/9511007

@inproceedings{resnik1995using,
  title = {{Using Information Content to Evaluate Semantic Similarity in a Taxonomy}},
  author = {Resnik, Philip},
  booktitle = {IJCAI-95},
  year = {1995},
  eprint = {cmp-lg/9511007},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/cmp-lg/9511007}
}

References

19 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Collins and E. Loftus. A spreading activation theory of semantic processing. Psychological Review, 82:407-428, 1975. find
  2. W. N. Francis and H. Kucera. Frequency Analysts of English Usage: Lexicon and Grammar. Houghton Mifflin, 1982. find
  3. Claudia Leacock and Martin Chodorow. Filling in a sparse training space for word sense identification, ms., March 1994. find
  4. Joon Ho Lee, Myoung Ho Kim, and Yoon Joon Lee. Information retrieval based on conceptual distance in IS-A hierarchies. Journal of Doc-umentatton, 49(2):188-207, June 1993. find
  5. Michael Lesk. Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone. In Proceedings of the 1986 SIGDOC Conference, pages 24-26, 1986. find
  6. George A. Miller and Walter G. Charles. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1- 28, 1991. find
  7. George Miller. WordNet: An on-line lexical database. International Journal of Lexicography, 3(4), 1990. (Special Issue). find
  8. M. Ross Quillian. Semantic memory. In M. Minsky, editor, Semantic Information Processing. MIT Press, Cambridge, MA, 1968. find
  9. Roy Rada and Ellen Bicknell. Ranking documents with a thesaurus. JASI6, 40(5):304-310, September 1989. find
  10. Roy Rada, Hafedh Mili, Ellen Bicknell, and Maria Blettner. Development and application of a metric on semantic nets. IEEE Transaction on Systems, Man, and Cybernetics, 19(1): 17-30, February 1989. find
  11. Philip Resnik. Selection and Information: A Class-Based Approach to Lexical Relationships. PhD thesis, University of Pennsylvania, December 1993. find
  12. Philip Resnik. Semantic classes and syntactic ambiguity. In Proceedings of the J993 ARPA Human Language Technology Workshop. Morgan Kaufmann, March 1993. find
  13. Philip Resnik. Disambiguating noun groupings with respect to WordNet senses. In Third Workshop on Very Large Corpora. Association for Computational Linguistics, 1995. find
  14. Sheldon Ross. A First Course in Probability. Macmillan, 1976. find
  15. Herbert Rubenstein and John Goodenough. Contextual correlates of synonymy. CACM, 8(10):627-633, October 1965. find
  16. John Sinclair (ed.). Collins COBUILD English Language Dictionary. Collins: London, 1987. find
  17. Michael Sussna. Word sense disambiguation for free-text indexing using a massive semantic network. In Proceedings of the Second International Conference on Information and Knowledge Management (CIKM-9S), Arlington, Virginia, 1993. find
  18. A. Tversky. Features of similarity. Psychological Review, 84:327 352, 1977. find
  19. Sholom M. Weiss and Casimir A. Kulikowski. Computer systems that learn: classification and prediction methods from statistics, neural nets, machine learning, and expert systems. Morgan Kaufmann, San Mateo, CA, 1991. find