Elamur Research⌕
☰
home / papers / 2010 / A Latent Dirichlet Allocation Method for Selectional Preferences

A Latent Dirichlet Allocation Method for Selectional Preferences

Paper · 2010 · Alan Ritter, Mausam and Oren Etzioni

Summary

Selectional preferences describe which argument values a relation typically accepts, such as companies as subjects of "is headquartered in." Class-based methods built on WordNet give readable classes but suffer from gaps in coverage and from word ambiguity. Similarity-based and discriminative methods predict better but do not produce explicit classes. The authors propose LDA-SP, which treats each relation as a document whose words are its observed arguments. They compare three topic models: IndependentLDA, JointLDA and LinkLDA. LinkLDA performs best because it models both arguments together while keeping separate topic sets for each. Inference uses collapsed Gibbs sampling. The training data are binary relation tuples harvested from the Web by TextRunner. On pseudo-disambiguation, LDA-SP clearly beats strong baselines. It also filters faulty inference-rule applications better than Pantel et al.'s ISP systems. Because its latent topics act as interpretable classes, it offers both predictive strength and transparency. It also yields a reusable preference repository.

Key points

  • At precision 0.9 on pseudo-disambiguation, LDA-SP achieves about 85% higher recall than the mutual-information approach of Erk (2007).
  • In inference-rule filtering, it surpasses ISP.IIM-∨ in both precision and recall, compared using CBC clusters (Pantel, 2003). It matches the high-precision point of ISP.JIM while offering much greater recall. KL divergence between relation topic distributions was also used to rank 26,000 rules.
  • One author spent roughly 4 hours mapping the 600 topics (300 per argument) to WordNet classes semi-automatically. The resulting class preferences for 100 unseen relations reached about 0.88 accuracy, compared with roughly 0.55 for Pantel's repository. The authors describe these results as preliminary.
  • A preference repository covering 10,000 relations was released at cs.washington.edu/research/ldasp.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of WashingtonSeattle · United States353 works · 200,043 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Topic modeling · 47 works

Cite

Cite as Alan Ritter, Mausam and Oren Etzioni. 2010. A Latent Dirichlet Allocation Method for Selectional Preferences. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ritter2010latent,
  title = {{A Latent Dirichlet Allocation Method for Selectional Preferences}},
  author = {Alan Ritter, Mausam and Oren Etzioni},
  booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/P10-1044/}
}

References

35 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michele Banko and Oren Etzioni. 2008. The tradeoffs between open and traditional relation extraction. In ACL-08: HLT. find
  2. Shane Bergsma, Dekang Lin, and Randy Goebel. 2008. Discriminative learning of selectional preference from unlabeled text. In EMNLP. find
  3. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. J. Mach. Learn. Res.
  4. Samuel Brody and Mirella Lapata. 2009. Bayesian word sense induction. In EACL, pages 103–111, Morristown, NJ, USA. Association for Computational Linguistics.
  5. Andrew Carlson, Justin Betteridge, Richard C. Wang, Estevam R. Hruschka Jr., and Tom M. Mitchell. 2010. Coupled semi-supervised learning for information extraction. In WSDM 2010. find
  6. Harr Chen, S. R. K. Branavan, Regina Barzilay, and David R. Karger. 2009. Global models of document structure using latent permutations. In NAACL. find
  7. Stephen Clark and David Weir. 2002. Class-based probability estimation using a semantic hierarchy. Comput. Linguist. find
  8. Ido Dagan, Lillian Lee, and Fernando C. N. Pereira. 1999. Similarity-based models of word cooccurrence probabilities. In Machine Learning. find
  9. Hal Daumé III and Daniel Marcu. 2006. Bayesian query-focused summarization. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics. find
  10. Hal Daume III. 2007. hbc: Hierarchical bayes compiler. http://hal3.name/hbc. ↗
  11. Katrin Erk. 2007. A simple, similarity-based model for selectional preferences. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. find
  12. Elena Erosheva, Stephen Fienberg, and John Lafferty. 2004. Mixed-membership models of scientific publications. Proceedings of the National Academy of Sciences of the United States of America. find
  13. Oren Etzioni, Michael Cafarella, Doug Downey, Ana maria Popescu, Tal Shaked, Stephen Soderl, Daniel S. Weld, and Alex Yates. 2005. Unsupervised named-entity extraction from the web: An experimental study. Artificial Intelligence. find
  14. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Comput. Linguist.
  15. T. L. Griffiths and M. Steyvers. 2004. Finding scientific topics. Proc Natl Acad Sci U S A. find
  16. Frank Keller and Mirella Lapata. 2003. Using the web to obtain frequencies for unseen bigrams. Comput. Linguist.
  17. Zornitsa Kozareva, Ellen Riloff, and Eduard Hovy. 2008. Semantic class learning from the web with hyponym pattern linkage graphs. In ACL-08: HLT.
  18. Hang Li and Naoki Abe. 1998. Generalizing case frames using a thesaurus and the mdl principle. Comput. Linguist.
  19. Dekang Lin and Patrick Pantel. 2001. Dirt-discovery of inference rules from text. In KDD. find
  20. Dekang Lin. 1998. Dependency-based evaluation of minipar. In Proc. Workshop on the Evaluation of Parsing Systems. find
  21. Qiaozhu Mei, Xuehua Shen, and ChengXiang Zhai. 2007. Automatic labeling of multinomial topic models. In KDD. find
  22. David Mimno, Hanna M. Wallach, Jason Naradowsky, David A. Smith, and Andrew McCallum. 2009. Polylingual topic models. In EMNLP.
  23. David Newman, Arthur Asuncion, Padhraic Smyth, and Max Welling. 2009. Distributed algorithms for topic models. JMLR. find
  24. David Newman, Jey Han Lau, Karl Grieser, and Timothy Baldwin. 2010. Automatic evaluation of topic coherence. In NAACL-HLT.
  25. Diarmuid Ó Séaghdha. 2010. Latent variable models of selectional preference. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.
  26. Patrick Pantel, Rahul Bhagat, Bonaventura Coppola, Timothy Chklovski, and Eduard H. Hovy. 2007. Isp: Learning inferential selectional preferences. In HLT-NAACL. find
  27. Patrick Andre Pantel. 2003. Clustering by committee. Ph.D. thesis, University of Alberta, Edmonton, Alta., Canada. find
  28. Joseph Reisinger and Marius Pasca. 2009. Latent variable models of concept-attribute attachment. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. find
  29. P. Resnik. 1996. Selectional constraints: an information-theoretic model and its computational realization. Cognition. find
  30. Philip Resnik. 1997. Selectional preference and sense disambiguation. In Proc. of the ACL SIGLEX Workshop on Tagging Text with Lexical Semantics: Why, What, and How? find
  31. Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1999. Inducing a semantically annotated lexicon via em-based clustering. In Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics.
  32. Lenhart Schubert and Matthew Tong. 2003. Extracting and evaluating general world knowledge from the brown corpus. In In Proc. of the HLT-NAACL Workshop on Text Meaning, pages 7–13.
  33. Benjamin Van Durme and Daniel Gildea. 2009. Topic models for corpus-centric knowledge generalization. In Technical Report TR-946, Department of Computer Science, University of Rochester, Rochester. find
  34. Tae Yano, William W. Cohen, and Noah A. Smith. 2009. Predicting response to political blog posts with topic models. In NAACL. find
  35. L. Yao, D. Mimno, and A. Mccallum. 2009. Efficient methods for topic model inference on streaming document collections. In KDD. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ritter-2010-latent-dirichlet-allocation-method-selectional/