Elamur Research⌕
☰
home / papers / 2010 / Latent Variable Models of Selectional Preference

Latent Variable Models of Selectional Preference

Paper · 2010 · Ó Séaghdha, Diarmuid

Summary

This work tackles selectional preference induction: estimating how plausible a given argument is for a predicate's slot, such as the object of a verb. The core difficulty is data sparsity, since a rarely observed pair may be either impossible or simply uncommon. The author borrows Bayesian topic models, built for document–word co-occurrence, and reworks them for predicate–argument data. Three LDA-related models are proposed: plain LDA, ROOTH-LDA (a Bayesian take on the latent-class model of Rooth et al., 1999) and DUAL-LDA. Each learns semantic classes without a hand-built taxonomy like WordNet, and it does so jointly with each predicate's distribution over those classes. Estimation uses Gibbs sampling with Dirichlet priors. The models are tested against human plausibility judgements. They match or beat ROOTH-EM, the vector-space model of Padó et al., and Web-count approaches. They are strongest on infrequent or unseen combinations, despite using much less data than Web-scale methods. This matters for languages and domains that lack huge corpora or semantic lexicons.

Key points

  • Evaluation uses the Keller and Lapata plausibility datasets, covering verb-object, noun-noun and adjective-noun pairs, each split into Seen and Unseen items.
  • Counting significance results, ROOTH-LDA wins most often, ahead of LDA and then ROOTH-EM. The author credits the Bayesian models' edge over EM mainly to smoothing by the Dirichlet priors.
  • ROOTH-EM overfits sharply beyond about 50 classes. LDA performs roughly the same for Z = 50–200. Switching off hyperparameter re-estimation lowers correlation by only about 1–2 points.
  • On the Holmes et al. (1989) set of 16 verbs, LDA and ROOTH-LDA perfectly separate plausible from implausible objects. They were trained on BNC verb-object data, roughly four times smaller than the 3GB AQUAINT corpus behind Bergsma et al.'s SVM model.
  • Proposed future work includes hierarchical priors to fold in raw counts for frequent pairs, combining the models with discriminative classifiers, and applications to semantic role labelling and FrameNet roles.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Topic modeling · 47 works

Cite

Cite as Ó Séaghdha, Diarmuid. 2010. Latent Variable Models of Selectional Preference. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{diarmuid2010latent,
  title = {{Latent Variable Models of Selectional Preference}},
  author = {Ó Séaghdha, Diarmuid},
  booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/P10-1045/}
}

References

33 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shane Bergsma, Dekang Lin, and Randy Goebel. 2008. Discriminative learning of selectional preferences from unlabeled text. In Proceedings of EMNLP-08, Honolulu, HI. find
  2. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993–1022.
  3. Jordan Boyd-Graber, David Blei, and Xiaojin Zhu. 2007. A topic model for word sense disambiguation. In Proceedings of EMNLP-CoNLL-07, Prague, Czech Republic. find
  4. Ted Briscoe, John Carroll, and Rebecca Watson. 2006. The second release of the RASP system. In Proceedings of the ACL-06 Interactive Presentation Sessions, Sydney, Australia.
  5. Samuel Brody and Mirella Lapata. 2009. Bayesian word sense induction. In Proceedings of EACL-09, Athens, Greece.
  6. Lou Burnard, 1995. Users’ Guide for the British National Corpus. British National Corpus Consortium, Oxford University Computing Service, Oxford, UK. find
  7. Jonathan Chang, Jordan Boyd-Graber, Sean Gerrish, Chong Wang, and David M. Blei. 2009. Reading tea leaves: How humans interpret topic models. In Proceedings of NIPS-09, Vancouver, BC.
  8. Stephen Clark and David Weir. 2002. Class-based probability estimation using a semantic hierarchy. Computational Linguistics, 28(2):187–206. find
  9. Katrin Erk. 2007. A simple, similarity-based model for selectional preferences. In Proceedings of ACL-07, Prague, Czech Republic. find
  10. Jenny Rose Finkel, Trond Grenager, and Christopher D. Manning. 2007. The infinite tree. In Proceedings of ACL-07, Prague, Czech Republic. find
  11. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288.
  12. Sharon Goldwater, Thomas L. Griffiths, and Mark Johnson. 2009. A Bayesian framework for word segmentation: Exploring the effects of context. Cognition, 112(1):21–54. find
  13. Thomas L. Griffiths and Mark Steyvers. 2004. Finding scientific topics. Proceedings of the National Academy of Sciences, 101(suppl. 1):5228–5235. find
  14. Thomas L. Griffiths, Mark Steyvers, and Joshua B. Tenenbaum. 2007. Topics in semantic representation. Psychological Review, 114(2):211–244. find
  15. Virginia M. Holmes, Laurie Stowe, and Linda Cupples. 1989. Lexical expectations in parsing complement-verb sentences. Journal of Memory and Language, 28(6):668–689. find
  16. Frank Keller and Mirella Lapata. 2003. Using the Web to obtain frequencies for unseen bigrams. Computational Linguistics, 29(3):459–484.
  17. Wei-Hao Lin, Theresa Wilson, Janyce Wiebe, and Alexander Hauptmann. 2006. Which side are you on? Identifying perspectives at the document and sentence levels. In Proceedings of CoNLL-06, New York, NY. find
  18. Xiao-Li Meng, Robert Rosenthal, and Donald B. Rubin. 1992. Comparing correlated correlation coefficients. Psychological Bulletin, 111(1):172–175. find
  19. Sebastian Padó, Ulrike Padó, and Katrin Erk. 2007. Flexible, corpus-based modelling of human plausibility judgements. In Proceedings of EMNLP-CoNLL-07, Prague, Czech Republic. find
  20. Patrick Pantel, Rahul Bhagat, Bonaventura Coppola, Timothy Chklovski, and Eduard Hovy. 2007. ISP: Learning inferential selectional preferences. In Proceedings of NAACL-HLT-07, Rochester, NY. find
  21. Keith Rayner, Tessa Warren, Barbara J. Juhasz, and Simon P. Liversedge. 2004. The effect of plausibility on eye movements in reading. Journal of Experimental Psychology: Learning Memory and Cognition, 30(6):1290–1301. find
  22. Joseph Reisinger and Marius Paşca. 2009. Latent variable models of concept-attribute attachment. In Proceedings of ACL-IJCNLP-09, Singapore. find
  23. Philip S. Resnik. 1993. Selection and Information: A Class-Based Approach to Lexical Relationships. Ph.D. thesis, University of Pennsylvania. find
  24. Alan Ritter, Mausam, and Oren Etzioni. 2010. A Latent Dirichlet Allocation method for selectional preferences. In Proceedings of ACL-10, Uppsala, Sweden.
  25. Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1999. Inducing a semantically annotated lexicon via EM-based clustering. In Proceedings of ACL-99, College Park, MD.
  26. Sabine Schulte im Walde, Christian Hying, Christian Scheible, and Helmut Schmid. 2008. Combining EM training and the MDL principle for an automatic verb classification incorporating selectional preferences. In Proceedings of ACL-08:HLT, Columbus, OH. find
  27. Yee W. Teh, Michael I. Jordan, Matthew J. Beal, and David M. Blei. 2006. Hierarchical Dirichlet processes. Journal of the American Statistical Association, 101(476):1566–1581. find
  28. Hanna Wallach, David Mimno, and Andrew McCallum. 2009. Rethinking LDA: Why priors matter. In Proceedings of NIPS-09, Vancouver, BC. find
  29. Hanna Wallach. 2006. Topic modeling: Beyond bag-of-words. In Proceedings of ICML-06, Pittsburgh, PA. find
  30. Yorick Wilks. 1978. Making preferences more active. Artificial Intelligence, 11:197–225. find
  31. Limin Yao, David Mimno, and Andrew McCallum. 2009. Efficient methods for topic model inference on streaming document collections. In Proceedings of KDD-09, Paris, France. find
  32. Beñat Zapirain, Eneko Agirre, and Lluı́s Màrquez. 2009. Generalizing over lexical features: Selectional preferences for semantic role classification. In Proceedings of ACL-IJCNLP-09, Singapore. find
  33. Huibin Zhang, Mingjie Zhu, Shuming Shi, and Ji-Rong Wen. 2009. Employing topic models for pattern-based semantic class discovery. In Proceedings of ACL-IJCNLP-09, Singapore. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/seaghdha-2010-latent-variable-models-selectional-preference/