Elamur Research⌕
☰
home / papers / 2002 / Class-Based Probability Estimation Using a Semantic Hierarchy

Class-Based Probability Estimation Using a Semantic Hierarchy

Paper · 2002 · Stephen Clark and David Weir

Summary

Clark and Weir address data sparsity in estimating how likely a given noun sense is to fill a specific argument slot of a predicate, for example the subject of run. Such estimates support PP-attachment and other structural disambiguation, parsing, WSD, anaphora resolution and language modeling. The authors use the noun hierarchy of WordNet 1.6. Bayes' theorem rewrites the target quantity so that the verb is conditioned on a class of senses (a hypernym together with everything below it), not on a single sense. A sense is then generalized upward through its hypernyms until a chi-square test signals that the verb's conditional probability differs significantly across the children of the current node. On a pseudo-disambiguation task, this approach beats two rival class-based estimators: Li and Abe's MDL-based generalization and Resnik's selectional-preference measure. Error analysis suggests both rivals generalize too aggressively. The work targets practical probability estimation for NLP systems rather than recovering linguistically ideal selectional restrictions.

Key points

  • The resource is the WordNet 1.6 noun hierarchy. It has roughly 66,000 concepts in nine subhierarchies, joined here under an added common root. It is a DAG, but only about 1% of nodes have multiple parents, so it is treated as nearly a tree.
  • The approximation uses a class-conditioned estimate, for example p(run | ⟨animal⟩, subj), in place of p(run | ⟨dog⟩, subj). This is combined with relative-frequency estimates of p(c | r) and p(v | r). All estimates are unsmoothed.
  • Two variants of the chi-square test are compared for the generalization step: the standard Pearson statistic and the log-likelihood statistic.
  • The baselines are Li and Abe's MDL approach (MDL software provided by Diana McCarthy) and Resnik's measure. The proposed method outperforms both on pseudo-disambiguation.
  • The article extends the authors' NAACL 2001 paper. Training data on predicate arguments is assumed to come from a treebank or shallow parser.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom381 works · 80,747 citations🎓University of SussexBrighton · United Kingdom24 works · 3,604 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Stephen Clark and David Weir. 2002. Class-Based Probability Estimation Using a Semantic Hierarchy. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/089120102760173643

@article{clark2002class,
  title = {{Class-Based Probability Estimation Using a Semantic Hierarchy}},
  author = {Stephen Clark and David Weir},
  journal = {Computational Linguistics},
  year = {2002},
  doi = {10.1162/089120102760173643},
  url = {https://doi.org/10.1162/089120102760173643}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abney, Steven P. and Marc Light. 1999. Hiding a semantic hierarchy in a Markov model. In Proceedings of the ACL Workshop on Unsupervised Learning in Natural Language Processing, University of Maryland, College Park, pages 1–8. find
  2. Agirre, Eneko and David Martinez. 2001. Learning class-to-class selectional preferences. In Proceedings of the Fifth ACL Workshop on Computational Language Learning, Toulouse, France, pages 15–22. find
  3. Agresti, Alan. 1996. An Introduction to Categorical Data Analysis. Wiley. find
  4. Bikel, Daniel M. 2000. A statistical model for parsing and word-sense disambiguation. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 155–163, Hong Kong. find
  5. Briscoe, Ted and John Carroll. 1997. find
  6. Automatic extraction of subcategorization from corpora. In Proceedings of the Fifth ACL Conference on Applied Natural Language Processing, pages 356–363, Washington, DC.
  7. Church, Kenneth W. and Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22–29.
  8. Ciaramita, Massimiliano and Mark Johnson. 2000. Explaining away ambiguity: Learning verb selectional preference with Bayesian networks. In Proceedings of the 18th International Conference on Computational Linguistics, pages 187–193, Saarbrucken, Germany. find
  9. Clark, Stephen. 2001. Class-Based Statistical Models for Lexical Knowledge Acquisition. Ph.D. dissertation, University of Sussex. find
  10. Clark, Stephen and David Weir. 1999. An iterative approach to estimating frequencies over a semantic hierarchy. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 258–265, University of Maryland, College Park. find
  11. Clark, Stephen and David Weir. 2000. A class-based probabilistic approach to structural disambiguation. In Proceedings of the 18th International Conference on Computational Linguistics, pages 194–200, Saarbrucken, Germany. find
  12. Clark, Stephen and David Weir. 2001. Class-based probability estimation using a semantic hierarchy. In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics, pages 95–102, Pittsburgh.
  13. Cressie, Noel A. C. and Timothy R. C. Read. 1984. Multinomial goodness of fit tests. find
  14. Journal of the Royal Statistics Society Series B, 46:440–464. find
  15. Dunning, Ted. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(1):61–74.
  16. Fellbaum, Christiane, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press. find
  17. Li, Hang and Naoki Abe. 1996. Clustering words with the MDL principle. In Proceedings of the 16th International Conference on Computational Linguistics, pages 4–9, Copenhagen, Denmark. find
  18. Li, Hang and Naoki Abe. 1998. Generalizing case frames using a thesaurus and the MDL principle. Computational Linguistics, 24(2):217–244.
  19. McCarthy, Diana. 1997. Word sense disambiguation for acquisition of selectional preferences. In Proceedings of the ACL/EACL Workshop on Automatic Information Extraction and Building of Lexical Semantic Resources for NLP Applications, pages 52–61, Madrid. find
  20. McCarthy, Diana. 2000. Using semantic preferences to identify verbal participation in role switching. In Proceedings of the First Conference of the North American Chapter of the Association for Computational Linguistics, pages 256–263, Seattle. find
  21. Miller, George A. 1998. Nouns in WordNet. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. MIT Press, pages 23–46. find
  22. Pedersen, Ted. 1996. Fishing for exactness. In Proceedings of the South-Central SAS Users Group Conference, Austin, pages 188–200. find
  23. Pedersen, Ted. 2001. A decision tree of bigrams is an accurate predictor of word sense. In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics, pages 79–86, Pittsburgh. find
  24. Pereira, Fernando, Naftali Tishby, and Lillian Lee. 1993. Distributional clustering of English words. In Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, pages 183–190, Columbus, OH.
  25. Resnik, Philip. 1993. Selection and Information: A Class-Based Approach to Lexical Relationships. Ph.D. dissertation, University of Pennsylvania. find
  26. Resnik, Philip. 1998. WordNet and class-based probabilities. In Christiane Fellbaum, editor, WordNet: An Electronic Lexical Database. MIT Press, pages 239–263. find
  27. Ribas, Francesc. 1995. On learning more appropriate selectional restrictions. In Proceedings of the Seventh Conference of the European Chapter of the Association for Computational Linguistics, pages 112–118, Dublin. find
  28. Rooth, Mats, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1999. Inducing a semantically annotated lexicon via EM-based clustering. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 104–111, University of Maryland, College Park.
  29. Wagner, Andreas. 2000. Enriching a lexical semantic net with selectional preferences by means of statistical corpus analysis. In Proceedings of the ECAI-2000 Workshop on Ontology Learning, Berlin, pages 37–42. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/clark-2002-class-based-probability-estimation-semantic/