Elamur Research⌕
☰
home / papers / 1994 / Using a Semantic Concordance for Sense Identification

Using a Semantic Concordance for Sense Identification

Summary

The paper asks what baseline accuracy an automatic word sense identification system must beat to count as progress. Gale, Church, and Yarowsky had suggested a 75% floor based on most-frequent senses without explaining how those frequencies were obtained. To ground such a floor in data, the authors use a semantic concordance in which every open-class word in 103 Brown Corpus passages is tagged with its WordNet 1.4 sense.

They compare three statistical strategies:

  • Random guessing among WordNet senses.
  • Most-frequent sense, with frequencies estimated from the concordance in a leave-one-passage-out setup.
  • Co-occurrence, which prefers senses that appeared in the same sentence as other words in the test sentence.

Knowing sense frequencies roughly doubles accuracy on polysemous words compared with guessing. Exploiting co-occurrence brings no further benefit, largely because the concordance is too small. The resulting figures, about 69% overall and 58% for polysemous words, offer realistic benchmarks for future disambiguation systems. They could also serve as fallback defaults in systems that use local context.

Key points

  • In WordNet, 82.3% of open-class words have a single sense. In running Brown Corpus text, however, only 27.2% of open-class tokens are monosemous.
  • Random guessing is right 45.0% of the time across 101,284 tokens, but only 26.8% on the 76,067 polysemous tokens.
  • The most-frequent heuristic was evaluated by holding out each of 104 passages in turn (the 103 concordance passages plus the newly tagged P7).
    • It averaged 66.9% overall and 56.4% on polysemous words.
    • Excluding forced guesses raised these to 69.0% and 58.2%.
  • The newly tagged test passage P7 (romance fiction) scored notably low at 62.5%. Across genres, fiction passages scored slightly higher than informative prose.
  • The co-occurrence heuristic reached 68.6% overall and 57.7% on polysemous words. For 28% of polysemous words, no co-occurrence data were available at all.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Institutions

🎓Princeton UniversityPrinceton · United States45 works · 34,022 citations

Cite

Cite as George A. Miller, Martin Chodorow, Shari Landes, Claudia Leacock and Robert G. Thomas. 1994. Using a Semantic Concordance for Sense Identification. Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{miller1994using,
  title = {{Using a Semantic Concordance for Sense Identification}},
  author = {George A. Miller, Martin Chodorow, Shari Landes, Claudia Leacock and Robert G. Thomas},
  booktitle = {Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994},
  year = {1994},
  doi = {10.3115/1075812.1075866},
  url = {https://doi.org/10.3115/1075812.1075866}
}

References

7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. I. Gale, W., Church, K. W., and Yarowsky, D. (1992) Estimating upper and lower beunds on the performance of word-sense disambiguation programs. Proceedings of the 30th Annual Meeting of the Association for ComputationaI Linguistics, pp. 249-256. 2. find
  2. Miller, G. A., Ed. (1990) Five Papers on WordNet. International Journal of Lexicology, 3, No. 4. (Revised, March 1993) 3. Francis, W. N., and Ku~era, H. (1982) Frequency Analysis of English Usage: Lexicon and Grammar. Boston, MA: Houghton Mifflin. 4. find
  3. Lorge, I. (1937) The English semantic count. Teachers College Record, 39, 65-77. 5. find
  4. Thomdike, E. L., and Barnhart, C. L., Eds. (1935) Thorndike-Barnhart Junior Dictionary. Glenview, IL: Scott Foresmam 6. find
  5. Collins COBUILD English Language Dictionary. (1987) London: Collins. 7. find
  6. Miller, G. A., Leacock, C., Tengi, R., and Bunker, R. (1993) A semantic concordance. Proceedings of a Human Language Technology Workshop, pp. 303-308. 8. Leacock, C., Towel], G., and Voorhees, E. (1993) Corpus-based statistical sense resolution. Proceedings of a Human Language Technology Workshop, p… our card
  7. Yarowsky, D. (1993) One sense per collocation. Proceedings of a Human Language Technology Workshop, pp. 266-271. find

This page in Markdown → https://md.research.elamur.ai/free/en/papers/miller-1994-semantic-concordance-sense-identification/