Elamur Research
☰
home / papers / 1995 / Unsupervised Word Sense Disambiguation Rivaling Supervised Methods

Unsupervised Word Sense Disambiguation Rivaling Supervised Methods

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/P95-1026.pdf

Summary

The paper addresses word sense disambiguation: choosing which dictionary sense an ambiguous English word has in a given context, without expensive hand-labelled training data. Yarowsky relies on two regularities of language. First, a word rarely switches sense within one document. Second, nearby words (collocations) are strong and consistent signals of sense. The method starts with a few seed collocations for each sense, such as life versus manufacturing for plant, and uses them to label a small part of an untagged corpus. It then trains a decision-list classifier, with evidence ranked by log-likelihood ratio, and applies it to the remaining examples. Each round adds the confidently labelled cases to the training set. A soft document-level constraint spreads and corrects labels. Tested on a set of ambiguous words, this bootstrapping reaches over 96% accuracy. That is roughly equal to a supervised decision list trained on the same contexts and better than Schütze's clustering-based unsupervised method. The work showed that raw text alone can give near-supervised results, and it became a standard template for self-training in NLP.

Key points

  • The "one sense per discourse" hypothesis was checked on 37,232 hand-tagged examples. When a word appears more than once in a document, it keeps the majority sense about 99.8% of the time. The constraint applies in about 50.1% of cases.
  • For adjacent content words, "one sense per collocation" is reliable more than 97% of the time. It weakens as the distance between words grows and is weaker for function words.
  • In the worked example for plant (7,538 instances), the initial seeds labelled only about 2% of the data. The other roughly 98% started out unlabelled.
  • The unsupervised system scored 95.5% versus 96.1% for the supervised one. With the discourse constraint added, it reached 96.5% and beat the supervised baseline.
  • On the same 4 words, it outperformed Schütze's unsupervised algorithm, 96.7% versus 92.2%, even though Schütze's data had a higher baseline.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: AI & machine learning

Topics: Word sense disambiguation

Cited by

455 papers of our library cite it (of the 140673 whose bibliographies we read); 13 of them have a page here, 442 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/981658.981684

@inproceedings{yarowsky1995unsupervised,
  title = {{Unsupervised Word Sense Disambiguation Rivaling Supervised Methods}},
  author = {Yarowsky, David},
  booktitle = {33rd Annual Meeting of the Association for Computational Linguistics},
  year = {1995},
  doi = {10.3115/981658.981684},
  url = {https://doi.org/10.3115/981658.981684}
}

References

25 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baum, L.E., "An Inequality and Associated Maximization Technique in Statistical Estimation of Probabilistic Functions of a Markov Process," Inequalities, v 3, pp 1-8, 1972. find
  2. Black, Ezra, "An Experiment in Computational Discrimination of English Word Senses," in IBM Journal of Research and Development, v 232, pp 185-194, 1988. find
  3. BriU, Eric, "A Corpus-Based Approach to Language Learning," Ph.D. Thesis, University of Pennsylvania, 1993. find
  4. Brown, Peter, Stephen Della Pietra, Vincent Della Pietra, and Robert Mercer, "Word Sense Disambiguation using Statistical Methods," Proceedings of the 29th Annual Meeting of the Association for Computational Linguistics, pp 264-270, 1991. find
  5. Bruce, Rebecca and Janyce Wiebe, "Word-Sense Disambiguation Using Decomposable Models," in Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, Las Cruces, NM, 1994. find
  6. Church, K.W., "A Stochastic Parts Program an Noun Phrase Parser for Unrestricted Text," in Proceeding, IEEE International Conference on Acoustics, Speech and Signal Processing, Glasgow, 1989. find
  7. Dagan, Ido and Alon Itai, "Word Sense Disambiguation Using a Second Language Monolingual Corpus", Computational Linguistics, v 20, pp 563-596, 1994. find
  8. Dempster, A.P., Laird, N.M, and Rubin, D.B., "Maximum Likelihood From Incomplete Data via the EM Algorithm," Journal of the Royal Statistical Society, v 39, pp 1-38, 1977. find
  9. Gale, W., K. Church, and D. Yarowsky, "A Method for Disambiguating Word Senses in a Large Corpus," Computers and the Humanities, 26, pp 415-439, 1992. find
  10. Gale, W., K. Church, and D. Yarowsky. "Discrimination Decisions for 100,000-Dimensional Spaces." In A. Zampoli, N. Calzolari and M. Palmer (eds.), Current Issues in Computational Linguistics: In Honour of Don Walker, Kluwer Academic Publishers, pp. 429-450, 1994. find
  11. Guthrie, J., L. Guthrie, Y. Wilks and H. Aidinejad, "Subject Dependent Co-occurrence and Word Sense Disambiguation," in Proceedings of the 29th Annual Meeting of the Association for Computational Linguistics, pp 146-152, 1991. find
  12. Hearst, Marti, "Noun Homograph Disambiguation Using Local Context in Large Text Corpora," in Using Corpora, University of Waterloo, Waterloo, Ontario, 1991. find
  13. Leacock, Claudia, Geoffrey Towell and Ellen Voorhees "Corpus-Based Statistical Sense Resolution," in Proceedings, ARPA Human Language Technology Workshop, 1993. find
  14. Lehman, Jill Fain, "Toward the Essential Nature of Statistical Knowledge in Sense Resolution", in Proceedings of the Twelfth National Conference on Artificial Intelligence, pp 734-471, 1994. find
  15. Lesk, Michael, "Automatic Sense Disambiguation: How to tell a Pine Cone from an Ice Cream Cone," Proceeding of the 1986 SIGDOC Conference, Association for Computing Machinery, New York, 1986. find
  16. Miller, George, "WordNet: An On-Line Lexical Database," International Journal of Lexicography, 3, 4, 1990. find
  17. Mosteller, Frederick, and David Wallace, Inference and Disputed Authorship: The Federalist, Addison-Wesley, Reading, Massachusetts, 1964. find
  18. Rivest, R. L., "Learning Decision Lists," in Machine Learning, 2, pp 229-246, 1987. find
  19. Schiitze, Hinrich, "Dimensions of Meaning," in Proceedings of Supercomputing '92, 1992. find
  20. Slator, Brian, "Using Context for Sense Preference," in Text-Based Intelligent Systems: Current Research in Text Analysis, Information Extraction and Retrieval, P.S. Jacobs, ed., GE Research and Development Center, Schenectady, New York, 1990. find
  21. Veronis, Jean and Nancy Ide, "Word Sense Disambiguation with Very Large Neural Networks Extracted from Machine Readable Dictionaries," in Proceedings, COLING-90, pp 389-394, 1990. find
  22. Yarowsky, David "Word-Sense Disambiguation Using Statistical Models of Roget's Categories Trained on Large Corpora," in Proceedings, COLING-92, Nantes, France, 1992. find
  23. Yaxowsky, David, "One Sense Per Collocation," in Proceedings, ARPA Human Language Technology Workshop, Princeton, 1993. find
  24. Yarowsky, David, "Decision Lists for Lexical Ambiguity Resolution: Application to Accent Restoration in Spanish and French," in Proceedings of the 32nd Annual Meeting of the Association .for Computational Linguistics, Las Cruces, NM, 1994. find
  25. Yarowsky, David. "Homograph Disambiguation in Speech Synthesis." In J. Hirschberg, R. Sproat and J. van Santen (eds.), Progress in Speech Synthesis, Springer-Verlag, to appear. find