Elamur Research⌕
☰
home / papers / 2010 / A Flexible, Corpus-Driven Model of Regular and Inverse Selectional Preferences

A Flexible, Corpus-Driven Model of Regular and Inverse Selectional Preferences

Paper · 2010 · Katrin Erk, Sebastian Padó and Ulrike Padó

Summary

The paper tackles the problem of learning selectional preferences, meaning how plausible a given noun is as the filler of a predicate's argument slot, without relying on hand-built resources such as WordNet. The authors present EPP, a fully distributional model. It scores a candidate headword by a weighted sum of its vector-space similarities to the headwords actually seen with that predicate and role. A central design choice separates two corpora. A primary corpus supplies the predicate–role–headword triples, while a larger generalization corpus supplies the vectors. This lets small, richly annotated data such as FrameNet be combined with the BNC. The model is tested on pseudo-disambiguation and on predicting human plausibility ratings. It is also applied to inverse preferences, where a noun predicts its likely verbs. EPP reaches error rates comparable to the WordNet-based RESNIK model with much broader coverage, and it beats the ROOTH ET AL. clustering model. Semantic role annotation in the primary corpus gives consistent gains. The work matters because it offers a cheap, resource-independent way to obtain broad-coverage plausibility estimates.

Key points

  • Sparsity motivates the work. In the 100-million-word BNC, the verb anglicize occurs with only nine direct objects, of which name is the sole one seen on more than a single occasion.
  • Setups compared:
    • FrameNet as primary corpus with the BNC for generalization (semantic roles).
    • The BNC alone (syntactic subjects and objects).
    • Dependency-based (DEPSPACE) vs. bag-of-words (WORDSPACE) vectors.
    • PCA reduction to 500 dimensions.
  • nGCM (a newly introduced exponential similarity), Lin, and Cosine perform best overall. nGCM suits sparse, fairly clean data, and Lin is more robust to noise. PCA brings Jaccard and Dice up to a similar level and removes a strong bias against rare arguments.
  • For inverse preferences (BNC, DISCR weighting), WORDSPACE with Lin reaches 27.2% error at 98.9% coverage. WORDSPACE beats DEPSPACE here because the confounders are verbs rather than nouns.
  • Weighting and space choice depend on the data:
    • Frequency weighting works for clean or dense data, while DISCR (a tf/idf variant) helps with sparse, noisy data.
    • DEPSPACEs pay off when many headwords are seen, while WORDSPACEs suit predicates with few headwords.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of Texas at AustinAustin · United States109 works · 56,942 citations🎓Heidelberg UniversityHeidelberg · Germany38 works · 41,356 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Lemma · 24 works

Cite

Cite as Katrin Erk, Sebastian Padó and Ulrike Padó. 2010. A Flexible, Corpus-Driven Model of Regular and Inverse Selectional Preferences. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00017

@article{erk2010flexible,
  title = {{A Flexible, Corpus-Driven Model of Regular and Inverse Selectional Preferences}},
  author = {Katrin Erk, Sebastian Padó and Ulrike Padó},
  journal = {Computational Linguistics},
  year = {2010},
  doi = {10.1162/coli\_a\_00017},
  url = {https://doi.org/10.1162/coli_a_00017}
}

References

73 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abe, Naoki and Hang Li. 1996. Learning word association norms using tree cut pair models. In Proceedings of the 10th International Conference on Machine Learning, pages 3–11, Bari. find
  2. Baroni, Marco, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide Web: A collection of very large linguistically processed Web-crawled corpora. Language Resources and Evaluation, 43(3):209–222. find
  3. Bergsma, Shane, Dekang Lin, and Randy Goebel. 2008. Discriminative learning of selectional preference from unlabeled text. In Proceedings of the 13th Conference on Empirical Methods in Natural Language Processing, pages 59–68, Honolulu, HI. find
  4. Best, John and D. E. Roberts. 1975. Algorithm AS 89: The upper tail probabilities of Spearman’s Rho. Applied Statistics, 24:377–379. find
  5. Briscoe, Ted and Bran Boguraev, editors. 1989. Computational Lexicography for Natural Language Processing. Longman Publishing Group, New York. find
  6. Brockmann, Carsten and Mirella Lapata. 2003. Evaluating and combining approaches to selectional preference acquisition. In Proceedings of the 16th Meeting of the European Chapter of the Association for Computational Linguistics, pages 27–34, Budapest. find
  7. Burnard, Lou, 1995. User’s Guide for the British National Corpus. British National Corpus Consortium, Oxford University Computing Services. find
  8. Ciaramita, Massimiliano and Mark Johnson. 2000. Explaining away ambiguity: Learning verb selectional preference with Bayesian networks. In Proceedings of the 18th International Conference on Computational Linguistics, pages 187–193, Saarbrücken. find
  9. Clark, Stephen and David Weir. 2001. Class-based probability estimation using a semantic hierarchy. In Proceedings of the 2nd Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 95–102, Pittsburgh, PA. find
  10. Collins, Michael. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, pages 16–23, Madrid.
  11. Curran, James. 2004. From Distributional to Semantic Similarity. Ph.D. thesis, University of Edinburgh. find
  12. Daelemans, Walter and Antal van den Bosch. 2005. Memory-Based Language Processing. Cambridge University Press, Cambridge, UK. find
  13. Dagan, Ido, Lillian Lee, and Fernando C. N. Pereira. 1999. Similarity-based models of word cooccurrence probabilities. Machine Learning, 34(1):34–69. find
  14. Dligach, Dmitriy and Martha Palmer. 2008. Novel semantic features for verb sense disambiguation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies, Short Papers, pages 29–32, Columbus, OH. find
  15. Dunning, Ted. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(1):61–74.
  16. Efron, Bradley and Robert Tibshirani. 1994. An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability 57. Chapman & Hall, London. find
  17. Erk, Katrin. 2007. A simple, similarity-based model for selectional preferences. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 216–223, Prague. find
  18. Erk, Katrin and Sebastian Padó. 2008. A structured vector space model for word meaning in context. In Proceedings of the 13th Conference on Empirical Methods in Natural Language Processing, pages 897–906, Honolulu, HI.
  19. Fillmore, Charles J., Christopher R. Johnson, and Miriam R. L. Petruck. 2003. Background to FrameNet. International Journal of Lexicography, 16:235–250. find
  20. Firth, John Rupert. 1957. A synopsis of linguistic theory, 1930–1955. In Philological Society, editor, Studies in Linguistic Analysis. Blackwell, Oxford, pages 1–32. find
  21. Garnsey, Susan, Neal Pearlmutter, Elizabeth Myers, and Melanie Lotocky. 1997. The contributions of verb bias and plausibility to the comprehension of temporarily ambiguous sentences. Journal of Memory and Language, 37:58–93. find
  22. Gildea, Daniel and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288.
  23. Grefenstette, Gregory. 1994. Explorations in Automatic Thesaurus Discovery. Kluwer Academic Publishers, Dordrecht. find
  24. Grishman, Ralph and John Sterling. 1992. Acquisition of selectional patterns. In Proceedings of the 14th International Conference on Computational Linguistics, pages 658–664, Nantes. find
  25. Harris, Zellig. 1968. Mathematical Structure of Language. Wiley, New York. find
  26. Hay, Jennifer, Aaron Nolan, and Katie Drager. 2006. From fush to feesh: Exemplar priming in speech perception. The Linguistic Review, 23(3):351–379. find
  27. Hindle, Donald. 1990. Noun classification from predicate-argument structures. In Proceedings of the 28th Annual Meeting of the Association for Computational Linguistics, pages 268–275, Pittsburgh, PA.
  28. Hindle, Donald and Mats Rooth. 1993. Structural ambiguity and lexical relations. Computational Linguistics, 19(1):103–120.
  29. Katz, Jerrold J. and Jerry A. Fodor. 1963. The structure of a semantic theory. Language, 39(2):170–210. find
  30. Katz, Jerrold J. and Paul M. Postal. 1964. An Integrated Theory of Linguistic Descriptions. Research Monograph No. 26. MIT Press, Cambridge, MA. find
  31. Konieczny, Lars and Philipp Döring. 2003. Anticipation of clause-final heads. Evidence from eye-tracking and SRNs. In Proceedings of the 4th International Conference on Cognitive Science, pages 330–335, Sydney. find
  32. Lakoff, George and Mark Johnson. 1980. Metaphors We Live By. University of Chicago Press, Chicago, IL. find
  33. Landauer, Thomas and Susan Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2):211–240. find
  34. Lee, Lillian. 1999. Measures of distributional similarity. In Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, pages 25–32, College Park, MA.
  35. Lin, Dekang. 1993. Principle-based parsing without overgeneration. In Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, pages 112–120, Columbus, OH. find
  36. Lin, Dekang. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the Joint Annual Meeting of the Association for Computational Linguistics and International Conference on Computational Linguistics, pages 768–774, Montreal.
  37. Lowe, Will. 2001. Towards a theory of semantic space. In Proceedings of the 23rd Annual Conference of the Cognitive Science Society, pages 576–581, Edinburgh. find
  38. Lowe, Will and Scott McDonald. 2000. The direct route: Mediated priming in semantic space. In Proceedings of the 22nd Annual Conference of the Cognitive Science Society, pages 675–680, Philadelphia, PA. find
  39. Lund, Kevin and Curt Burgess. 1996. Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, and Computers, 28:203–208. find
  40. McCarthy, Diana. 2000. Using semantic preferences to identify verbal participation in role switching alternations. In Proceedings of the 1st Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 256–263, Seattle, WA. find
  41. McCarthy, Diana and John Carroll. 2003. Disambiguating nouns, verbs, and adjectives using automatically acquired selectional preferences. Computational Linguistics, 29(4):639–654. find
  42. McCarthy, Diana, Rob Koeling, Julie Weeds, and John Carroll. 2004. Finding predominant word senses in untagged text. In Proceedings of the 42th Annual Meeting of the Association for Computational Linguistics, pages 279–286, Barcelona.
  43. McCarthy, Diana, Sriram Venkatapathy, and Aravind K. Joshi. 2007. Detecting compositionality of verb-object combinations using selectional preferences. In Proceedings of the 12th Joint Conference on Empirical Methods in Natural Language Processing and Conference on Natural Language Learning, pages 3… find
  44. McDonald, Scott and Chris Brew. 2004. A distributional model of semantic context effects in lexical processing. In Proceedings of the 42th Annual Meeting of the Association for Computational Linguistics, pages 17–24, Barcelona. find
  45. McRae, Ken, Todd Ferretti, and Liane Amyote. 1997. Thematic roles as verb-specific concepts. Language and Cognitive Processes, 12(2/3):137–176. find
  46. McRae, Ken, Mary Hare, Jeffrey Elman, and Todd Ferretti. 2005. A basis for generating expectancies for verbs from nouns. Memory and Cognition, 33(7):1174–1184. find
  47. McRae, Ken, Mary Hare, Todd Ferretti, and Jeffrey Elman. 2001. Activating verbs typical agents, patients, instruments, and locations via event schemas. In Proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society, pages 617–622, Mahwah, NJ. find
  48. McRae, Ken, Michael Spivey-Knowlton, and Michael Tanenhaus. 1998. Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38:283–312. find
  49. Miller, George, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller. 1990. Five papers on WordNet. International Journal of Lexicography, 3(4):235–312. find
  50. Nakatani, Kentaro and Edward Gibson. 2009. An on-line study of Japanese nesting complexity. Cognitive Science, 1(34):94–112. find
  51. Nastase, Vivi. 2008. Unsupervised all-words word sense disambiguation with grammatical dependencies. In Proceedings of the 3rd International Joint Conference on Natural Language Processing, pages 757–762, Honolulu, HI. find
  52. Nosofsky, Robert. 1986. Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115(1):39–57. find
  53. Padó, Sebastian and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161–199.
  54. Padó, Sebastian, Ulrike Padó, and Katrin Erk. 2007. Flexible, corpus-based modelling of human plausibility judgements. In Proceedings of the 12th Joint Conference on Empirical Methods in Natural Language Processing and Conference on Natural Language Learning, pages 400–409, Prague. find
  55. Padó, Ulrike. 2007. The Integration of Syntax and Semantic Plausibility in a Wide-Coverage Model of Human Sentence Processing. Ph.D. thesis, Saarland University, Saarbrücken, Germany. find
  56. Padó, Ulrike, Matthew W. Crocker, and Frank Keller. 2009. A probabilistic model of semantic plausibility in sentence processing. Cognitive Science, 33:794–838. find
  57. Pantel, Patrick, Rahul Bhagat, Bonaventura Coppola, Timothy Chklovski, and Eduard Hovy. 2007. ISP: Learning inferential selectional preferences. In Proceedings of the Joint Human Language Technology Conference and Annual Meeting of the North American Chapter of the Association for Computational Ling… find
  58. Pereira, Fernando, Naftali Tishby, and Lillian Lee. 1993. Distributional clustering of English words. In Proceedings of the 31st Annual Meeting of the Association for Computational Linguistics, pages 183–190, Columbus, OH.
  59. Pustejovsky, James. 1995. The Generative Lexicon. MIT Press, Cambridge, MA. find
  60. Resnik, Philip. 1996. Selectional constraints: An information-theoretic model and its computational realization. Cognition, 61:127–159. find
  61. Rooth, Mats, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1999. Inducing a semantically annotated lexicon via EM-based clustering. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 104–111, College Park, MA.
  62. Salton, Gerard, Anita Wong, and Chung-Shu Yang. 1975. A vector-space model for information retrieval. Journal of the American Society for Information Science, 18:613–620. find
  63. Schulte im Walde, Sabine. 2010. Comparing computational approaches to selectional preferences: Second-order co-occurrence vs. latent semantic clusters. In Proceedings of the 7th International Conference on Language Resources and Evaluation, pages 1381–1388, Valleta. find
  64. Schulte im Walde, Sabine, Christian Hying, Christian Scheible, and Helmut Schmid. 2008. Combining EM training and the MDL principle for an automatic verb classification incorporating selectional preferences. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics,… find
  65. Shepard, Roger. 1987. Towards a universal law of generalization for psychological science. Science, 237(4820):1317–1323. find
  66. Stolcke, Andreas. 2002. SRILM—an extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, pages 901–904, Denver, CO. find
  67. Toutanova, Kristina, Christoper D. Manning, Dan Flickinger, and Stephan Oepen. 2005. Stochastic HPSG parse selection using the Redwoods corpus. Journal of Research on Language and Computation, 3(1):83–105. find
  68. Trueswell, John, Michael Tanenhaus, and Susan Garnsey. 1994. Semantic influences on parsing: Use of thematic role information in syntactic ambiguity resolution. Journal of Memory and Language, 33:285–318. find
  69. Vandekerckhove, Bram, Dominiek Sandra, and Walter Daelemans. 2009. A robust and extensible exemplar-based model of thematic fit. In Proceedings of the 12th Meeting of the European Chapter of the Association for Computational Linguistics, pages 826–834, Athens. find
  70. Wilks, Yorick. 1975. Preference semantics. In E. Keenan, editor, Formal Semantics of Natural Language. Cambridge University Press, Cambridge, UK, pages 329–350. find
  71. Yarowsky, David. 1993. One sense per collocation. In Proceedings of the ARPA Human Language Technology Workshop, pages 266–271, Princeton, NJ.
  72. Zanzotto, Fabio Massimo, Marco Pennacchiotti, and Maria Teresa Pazienza. 2006. Discovering asymmetric entailment relations between verbs using selectional preferences. In Proceedings of the Joint Annual Meeting of the Association for Computational Linguistics and International Conference on Computat… find
  73. Zapirain, Beñat, Eneko Agirre, and Lluís Màrquez. 2009. Generalizing over lexical features: Selectional preferences for semantic role classification. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics, pages 73–76, Singapore. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/erk-2010-flexible-corpus-driven-model-regular/