Elamur Research⌕
☰
home / papers / 2013 / Selectional Preferences for Semantic Role Classification

Selectional Preferences for Semantic Role Classification

Paper · 2013 · Beñat Zapirain, Eneko Agirre, Lluís Màrquez et al.

Summary

Classifying semantic roles depends heavily on lexical cues such as argument head words. These cues are sparse, which hurts accuracy, especially on text from a new domain. The authors address this by generalizing head words with automatically acquired selectional preferences (SP). They build several SP models from WordNet classes and from distributional-similarity thesauri. A new element is that they learn preferences for prepositions as well as for verbs. Experiments use the CoNLL-2005 data (PropBank), with Brown corpus text as the out-of-domain test.

On their own, the SP models clearly beat simple head-word matching. When their predictions are combined with the state-of-the-art SwiRL system, role classification errors drop substantially. Full semantic role labeling also improves slightly but significantly. A manual error analysis shows SPs help most where parse information is wrong or cannot settle the role. Their main weakness is that they ignore syntax and word position. The work shows that semantic generalization can reduce sparseness and domain-shift problems in SRL.

Key points

  • Standalone SP models outperform the lexical baseline by about 20 F1 points in domain and nearly 40 F1 points out of domain.
  • Distributional-similarity models beat WordNet-based ones, and second-order similarity variants perform best. Combining verb and preposition SPs beats verb-only SPs.
  • Adding SP-based features to SwiRL reduces classification errors by 17% in domain and 13% out of domain.
  • In end-to-end SRL, the gains are small but statistically significant, even though only about 4% of argument candidates are affected.
  • Some errors come from syntax-driven cases, such as the usual Arg2 after "fail", which SPs cannot capture. The authors propose SP models that also encode syntactic information as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of the Basque CountryLeioa · Spain57 works · 16,520 citations🎓Universitat Politècnica de CatalunyaBarcelona · Spain31 works · 15,636 citations🎓University of ArizonaTucson · United States38 works · 10,603 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semantic role labeling · 145 works

Cite

Cite as Beñat Zapirain, Eneko Agirre, Lluís Màrquez and Mihai Surdeanu. 2013. Selectional Preferences for Semantic Role Classification. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00145

@article{zapirain2013selectional,
  title = {{Selectional Preferences for Semantic Role Classification}},
  author = {Beñat Zapirain, Eneko Agirre, Lluís Màrquez and Mihai Surdeanu},
  journal = {Computational Linguistics},
  year = {2013},
  doi = {10.1162/coli\_a\_00145},
  url = {https://doi.org/10.1162/coli_a_00145}
}

References

57 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agirre, Eneko, Timothy Baldwin, and David Martinez. 2008. Improving parsing and PP attachment performance with sense information. In Proceedings of ACL-08: HLT, pages 317–325, Columbus, OH. find
  2. Agirre, Eneko, Kepa Bengoetxea, Koldo Gojenola, and Joakim Nivre. 2011. Improving dependency parsing with semantic classes. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 699–703, Portland, OR. find
  3. Agirre, Eneko and David Martinez. 2001. Learning class-to-class selectional preferences. In Proceedings of the 2001 Workshop on Computational Natural Language Learning (CoNLL-2001), pages 1–8, Toulouse. find
  4. Agirre, Eneko and German Rigau. 1996. Word sense disambiguation using conceptual density. In Proceedings of the 16th Conference on Computational Linguistics - Volume 1, COLING ’96, pages 16–22, Stroudsburg, PA. find
  5. Baroni, Marco and Alessandro Lenci. 2010. Distributional memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4):673–721.
  6. Bergsma, Shane, Dekang Lin, and Randy Goebel. 2008. Discriminative learning of selectional preference from unlabeled text. In Proceedings of EMNLP, pages 59–68, Honolulu, HI. find
  7. Boas, H. C. 2002. Bilingual framenet dictionaries for machine translation. In Proceedings of the Third International Conference on Language Resources and Evaluation (LREC), pages 1,364–1,371, Las Palmas de Gran Canaria.
  8. Brockmann, Carsten and Mirella Lapata. 2003. Evaluating and combining approaches to selectional preference acquisition. In Proceedings of the 10th Conference of the European Chapter of the Association of Computational Linguistics (EACL-2003), pages 27–34, Budapest. find
  9. Carreras, X. and L. Màrquez. 2004. Introduction to the CoNLL-2004 Shared Task: Semantic Role Labeling. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004), pages 89–97, Boston, MA. find
  10. Carreras, X. and L. Màrquez. 2005. Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), pages 152–164, Ann Arbor, MI.
  11. Chakraborti, Sutanu, Nirmalie Wiratunga, Robert Lothian, and Stuart Watt. 2007. Acquiring word similarities with higher order association mining. In Proceedings of the 7th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development, ICCBR ’07, pages 61–76, Berlin. find
  12. Chambers, Nathanael and Daniel Jurafsky. 2010. Improving the use of pseudo-words for evaluating selectional preferences. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 445–453, Uppsala, Sweden. find
  13. Charniak, E. 2000. A maximum-entropy inspired parser. In Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL-2000), pages 132–139, Seattle, WA.
  14. Clark, Stephen and Stephen Weir. 2002. Class-based probability estimation using a semantic hierarchy. Computational Linguistics, 28(2):187–206. find
  15. Edmonds, Philip. 1997. Choosing the word most typical in context using a lexical co-occurrence network. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics, ACL ’98,… find
  16. Erk, Katrin. 2007. A simple, similarity-based model for selectional preferences. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL-2007), pages 216–223, Prague. find
  17. Erk, Katrin, Sebastian Padó, and Ulrike Padó. 2010. A flexible, corpus-driven model of regular and inverse selectional preferences. Computational Linguistics, 36(4):723–763.
  18. Fillmore, C. J., J. Ruppenhofer, and C. F. Baker. 2004. FrameNet and representing the link between semantic and syntactic relations. In Frontiers in Linguistics, volume I of Language and Linguistics Monograph Series B. Institute of Linguistics, Academia Sinica, Taipei, pages 19–59. find
  19. Gildea, D. and D. Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288.
  20. Grefenstette, Gregory. 1992. Sextant: Exploring unexplored contexts for semantic extraction from syntactic analysis. In ACL’92, pages 324–326, Newark, DE. find
  21. Higashinaka, Ryuichiro and Hideki Isozaki. 2008. Corpus-based question answering for why-questions. In Proceedings of the Third International Joint Conference on Natural Language Processing (IJCNLP), pages 418–425, Hyderabad. find
  22. Hindle, Donald. 1990. Noun classification from predicate-argument structures. In Proceedings of the 28th Annual Meeting of the Association for Computational Linguistics (ACL-1990), pages 268–275, Pittsburgh, PA.
  23. Koo, Terry, Xavier Carreras, and Michael Collins. 2008. Simple semi-supervised dependency parsing. In Proceedings of ACL-08: HLT, pages 595–603, Columbus, OH.
  24. Lee, Lillian. 1999. Measures of distributional similarity. In 37th Annual Meeting of the Association for Computational Linguistics, pages 25–32, College Park, MD.
  25. Li, Hang and Naoki Abe. 1998. Generalizing case frames using a thesaurus and the MDL principle. Computational Linguistics, 24(2):217–244.
  26. Lin, Dekang. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (COLING-ACL-1998), pages 768–774, Montreal.
  27. Litkowski, K. C. and O. Hargraves. 2005. The preposition project. In Proceedings of the ACL-SIGSEM Workshop on the Linguistic Dimensions of Prepositions and their Use in Computational Linguistic Formalisms and Applications, pages 171–179, Colchester. find
  28. Litkowski, K. C. and O. Hargraves. 2007. SemEval-2007 Task 06: Word-sense disambiguation of prepositions. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), pages 24–29, Prague. find
  29. Litkowski, Ken and Orin Hargraves. 2006. Coverage and inheritance in the preposition project. In Prepositions ’06: Proceedings of the Third ACL-SIGSEM Workshop on Prepositions, pages 37–44, Trento. find
  30. Marcus, Mitchell, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: Annotating predicate argument structure. In Proceedings of the Workshop on Human Language Technology (HLT-94), pages 114–119, Plainsboro, NJ.
  31. Màrquez, Lluı́s, Xavier Carreras, Kenneth C. Litkowski, and Suzanne Stevenson. 2008. Semantic role labeling: An introduction to the special issue. Computational Linguistics, 34(2):145–159. find
  32. McCarthy, Diana and John Carroll. 2003. Disambiguating nouns, verbs, and adjectives using automatically acquired selectional preferences. Computational Linguisties, 29:639–654. find
  33. Melli, Gabor, Yang Wang, Yudong Liu, Mehdi M. Kashani, Zhongmin Shi, Baohua Gu, Anoop Sarkar, and Fred Popowich. 2005. Description of SQUASH, the SFU question answering summary handler for the DUC-2005 summarization task. In Proceedings of Document Understanding Workshop, HLT/EMNLP Annual Meeting, V… find
  34. Moschitti, Alessandro, Silvia Quarteroni, Roberto Basili, and Suresh Manandhar. 2007. Exploiting syntactic and shallow semantic kernels for question/answer classification. In Proceedings of the 45th Conference of the Association for Computational Linguistics (ACL), pages 776–783, Prague. find
  35. Narayanan, S. and S. Harabagiu. 2004. Question answering based on semantic structures. In Proceedings of the 20th International Conference on Computational Linguistics (COLING), pages 693–701, Geneva. find
  36. Noreen, E. W. 1989. Computer-Intensive Methods for Testing Hypotheses: An Introduction, Wiley. find
  37. Ó Séaghdha, Diarmuid. 2010. Latent variable models of selectional preference. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 435–444, Uppsala.
  38. Padó, Sebastian and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161–199.
  39. Padó, Sebastian, Ulrike Padó, and Katrin Erk. 2007. Flexible, corpus-based modelling of human plausibility judgements. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL-2007), pages 400–409, Prag… find
  40. Palmer, M., D. Gildea, and P. Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–105.
  41. Pantel, Patrick, Rahul Bhagat, Bonaventura Coppola, Timothy Chklovski, and Eduard Hovy. 2007. ISP: Learning inferential selectional preferences. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main… find
  42. Pantel, Patrick and Dekang Lin. 2000. An unsupervised approach to prepositional phrase attachment using contextually similar words. In Proceedings of the 38th Annual Conference of the Association of Computational Linguistics (ACL-2000), pages 101–108, Hong Kong.
  43. Pradhan, S., W. Ward, and J. H. Martin. 2008. Towards robust semantic role labeling. Computational Linguistics, 34(2):289–310.
  44. Rada, R., H. Mili, E. Bicknell, and M. Blettner. 1989. Development and application of a metric on semantic nets. IEEE Transactions on Systems, Man, and Cybernetics, 19(1):17–30. find
  45. Ratinov, Lev and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pages 147–155, Boulder, CO.
  46. Resnik, Philip. 1993a. Selection and Information: A Class-Based Approach to Lexical Relationships. Ph.D. thesis, University of Pennsylvania. find
  47. Resnik, Philip. 1993b. Semantic classes and syntactic ambiguity. In Proceedings of the Workshop on Human Language Technology, pages 278–283, Morristown, NJ. find
  48. Ritter, Alan, Mausam, and Oren Etzioni. 2010. A latent Dirichlet allocation method for selectional preferences. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 424–434, Uppsala.
  49. Schütze, Hinrich. 1998. Automatic word sense discrimination. Computational Linguistics, 24(1):97–123.
  50. Srikumar, V. and D. Roth. 2011. A joint model for extended semantic role labeling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 129–139, Edinburgh. find
  51. Surdeanu, M., S. Harabagiu, J. Williams, and P. Aarseth. 2003. Using predicate-argument structures for information extraction. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL-2003), pages 8–15, Sapporo.
  52. Surdeanu, Mihai, Massimiliano Ciaramita, and Hugo Zaragoza. 2011. Learning to rank answers to non-factoid questions from Web collections. Computational Linguistics, 37(2):351–383.
  53. Surdeanu, Mihai, Lluı́s Màrquez, Xavier Carreras, and Pere R. Comas. 2007. Combination strategies for semantic role labeling. Journal of Artificial Intelligence Research (JAIR), 29:105–151. find
  54. Sussna, Michael. 1993. Word sense disambiguation for free-text indexing using a massive semantic network. In Proceedings of the Second International Conference on Information and Knowledge Management, CIKM ’93, pages 67–74, New York, NY. find
  55. Wilks, Yorick. 1975. Preference semantics. In E. L. Kaenan, editor, Formal Semantics of Natural Language. Cambridge University Press, Cambridge, MA, pages 329–348. find
  56. Zapirain, Beñat, Eneko Agirre, and Lluı́s Màrquez. 2009. Generalizing over lexical features: Selectional preferences for semantic role classification. In Proceedings of the Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint… find
  57. Zapirain, Beñat, Eneko Agirre, Lluı́s Màrquez, and Mihai Surdeanu. 2010. Improving semantic role classification with selectional preferences. In Proceedings of the 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), pages 373–376,… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zapirain-2013-selectional-preferences-semantic-role-classification/