Elamur Research⌕
☰
home / papers / 2019 / Unsupervised Compositionality Prediction of Nominal Compounds

Unsupervised Compositionality Prediction of Nominal Compounds

Paper · 2019 · Silvio Cordeiro, Aline Villavicencio, Marco Idiart et al.

Summary

Nominal compounds range from transparent (olive oil) to idiomatic (snake oil), and NLP systems that build phrase meaning from word meanings struggle with the idiomatic end. The authors propose an unsupervised framework that uses distributional semantic models (DSMs) to estimate how compositional a compound is. The core assumption is that a compositional compound should have a vector close to a combination of its parts' vectors, produced by a composition function. Low similarity between the two signals idiomaticity. For evaluation, they built new datasets of human compositionality judgments for English, French and Portuguese. They then trained hundreds of DSMs with different model types, parameters, training corpora and composition functions, and compared their predictions to the human ratings. The predictions agreed strongly with the human ratings, which suggests DSMs encode idiomaticity. Across languages, morphological richness and corpus size affected performance, and weighting the two components equally gave the best results. The work provides cross-lingual resources and a systematic benchmark of compositionality prediction. This matters for tasks such as semantic parsing, word sense disambiguation and machine translation.

Key points

  • The study covers three languages (English, French, Portuguese), each with a newly collected dataset of human compositionality judgments. The listed compounds carry average scores on a 0–5 scale.
  • The authors built 684 DSMs and ran more than 9,072 analyses. These varied the model type, its configuration, the training corpus and the composition function.
  • Compositionality is predicted by comparing the compound's own vector with a vector composed from its head and modifier. The method needs no supervision.
  • In the cross-lingual analysis, morphological variation and corpus size influenced how well compositionality was predicted. Combining the two components uniformly worked best.
  • The article extends earlier work by Cordeiro, Ramisch, Villavicencio and colleagues (2016). It adds Portuguese, new composition functions and DSMs, and an analysis of the effect of corpus size.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🎓University of EssexColchester · United Kingdom34 works · 3,537 citations🎓Aix-Marseille UniversitéMarseille · France24 works · 39,160 citations🎓Universidade Federal do Rio Grande do SulPorto Alegre · Brazil8 works · 522 citations🔬Laboratoire d’Informatique et SystèmesMarseille · France8 works · 106 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Silvio Cordeiro, Aline Villavicencio, Marco Idiart and Carlos Ramisch. 2019. Unsupervised Compositionality Prediction of Nominal Compounds. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00341

@article{cordeiro2019unsupervised,
  title = {{Unsupervised Compositionality Prediction of Nominal Compounds}},
  author = {Silvio Cordeiro, Aline Villavicencio, Marco Idiart and Carlos Ramisch},
  journal = {Computational Linguistics},
  year = {2019},
  doi = {10.1162/coli\_a\_00341},
  url = {https://doi.org/10.1162/coli_a_00341}
}

References

72 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Agirre, Eneko, Enrique Alfonseca, Keith B. Hall, Jana Kravalova, Marius Pasca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and wordnet-based approaches. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational…
  2. Artstein, Ron, and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. find
  3. Baldwin, Timothy, and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of Natural Language Processing, 2nd edition. CRC Press, Taylor and Francis Group, Boca Raton, FL, pages 267–292. find
  4. Bannard, Colin, Timothy Baldwin, and Alex Lascarides. 2003. A statistical approach to the semantics of verb-particles. In Proceedings of the ACL 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment (Volume 18), pages 65–72, Stroudsburg, PA. find
  5. Baroni, Marco, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation, 43(3):209–226. Baroni, Marco, Georgiana Dinu, and Germán Kruszewski. 2014. Don’t count, pred… find
  6. Baroni, Marco, and Alessandro Lenci. 2010. Distributional memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4):673–721.
  7. Bick, Eckhard. 2000. The Parsing System “palavras”: Automatic Grammatical Analysis of Portuguese in a Constraint Grammar Framework. Ph.D. thesis, University of Aarhus. find
  8. Boos, Rodrigo, Kassius Prestes, and Aline Villavicencio. 2014. Identification of multiword expressions in the brWaC. In Proceedings of the Conference on Language Resources and Evaluation 2014, pages 728–735, ELRA. ACL Anthology Identifier: L14–1429. find
  9. Bride, Antoine, Tim Van de Cruys, and Nicholas Asher. 2015. A generalisation of lexical functions for composition in distributional semantics. In Association for Computational Linguistics (1), pages 281–291. find
  10. Bullinaria, John A., and Joseph P. Levy. 2012. Extracting semantic representations from word co-occurrence statistics: Stop-lists, stemming, and SVD. Behavior Research Methods, 44(3):890–907. Camacho-Collados, José, Mohammad Taher Pilehvar, and Roberto Navigli. 2015. A framework for the constructio… find
  11. Carpuat, Marine, and Mona Diab. 2010. Task-based evaluation of multiword expressions: A pilot study in statistical machine translation. In Proceedings of NAACL/HLT 2010, pages 242–245, Los Angeles. find
  12. Church, Kenneth Ward, and Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22–29.
  13. Cohen, Jacob. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1):37–46. Constant, Mathieu, Gülşen Eryiğit, Johanna Monti, Lonneke Van Der Plas, Carlos Ramisch, Michael Rosner, and Amalia Todirascu. 2017. Multiword expression processing: A survey.… find
  14. Cordeiro, Silvio, Carlos Ramisch, Marco Idiart, and Aline Villavicencio. 2016. Predicting the compositionality of nominal compounds: Giving word embeddings a hard time. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1986–1997… find
  15. Cordeiro, Silvio, Carlos Ramisch, and Aline Villavicencio. 2016. mwetoolkit+sem: Integrating word embeddings in the mwetoolkit for semantic MWE processing. find
  16. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 1221–1225, European Language Resources Association (ELRA), Paris. find
  17. Curran, James R., and Marc Moens. 2002. Scaling context space. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 231–238. find
  18. Deerwester, Scott, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, and Richard Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391. find
  19. Evert, Stefan. 2004. The Statistics of Word Cooccurrences: Word Pairs and Collocations. Ph.D. thesis, Institut für maschinelle Sprachverarbeitung, University of Stuttgart, Stuttgart, Germany. find
  20. Farahmand, Meghdad, Aaron Smith, and Joakim Nivre. 2015. A multiword expression data set: Annotating non-compositionality and conventionalization for English noun compounds. In Proceedings of the 11th Workshop on Multiword Expressions, pages 29–33, Association for Computational Linguistics, Denver. find
  21. Fazly, Afsaneh, Paul Cook, and Suzanne Stevenson. 2009. Unsupervised type and token identification of idiomatic expressions. Computational Linguistics, 35(1):61–103.
  22. Ferret, Olivier. 2013. Identifying bad semantic neighbors for improving distributional thesauri. In Association for Computational Linguistics (1), pages 561–571. find
  23. Finlayson, Mark, and Nidhi Kulkarni. 2011. Detecting multi-word expressions improves word sense disambiguation. In Proceedings of the Association for Computational Linguistics 2011 Workshop on MWEs, pages 20–24, Portland, OR. find
  24. Firth, John R. 1957. A synopsis of linguistic theory, 1930–1955. In F. R. Palmer, ed., Selected Papers of J. R. Firth, pages 168–205, Longman, London. find
  25. Fleiss, Joseph L., and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33(3):613–619. Frege, Gottlob. 1892/1960. Über sinn und bedeutung. Zeitschrift für Philosophie und philosoph… find
  26. Freitag, Dayne, Matthias Blume, John Byrnes, Edmond Chow, Sadik Kapadia, Richard Rohwer, and Zhiqiang Wang. 2005. New experiments in distributional representations of synonymy. In Proceedings of the Ninth Conference on Computational Natural Language Learning, pages 25–32. find
  27. Girju, Roxana, Dan Moldovan, Marta Tatu, and Daniel Antohe. 2005. On the semantics of noun compounds. Computer Speech & Language, 19(4):479–496. find
  28. Goldberg, Adele E. 2015. Compositionality, Chapter 24. find
  29. Routledge, Amsterdam. Guevara, Emiliano. 2011. Computing semantic compositionality in distributional semantics. In Proceedings of the Ninth International Conference on Computational Semantics, IWCS ’11, pages 135–144, Association for Computational Linguistics, Stroudsburg, PA. find
  30. Harris, Zellig. 1954. Distributional structure. Word, 10:146–162. find
  31. Hartung, Matthias, Fabian Kaupmann, Soufian Jebbara, and Philipp Cimiano. 2017. Learning compositionality functions on word embeddings for modelling attribute meaning in adjective-noun phrases. In Proceedings of the 15th Meeting of the European Chapter of the Association for Computational Linguistic… find
  32. Hwang, Jena D., Archna Bhatia, Clare Bonial, Aous Mansouri, Ashwini Vaidya, Nianwen Xue, and Martha Palmer. 2010. Propbank annotation of multilingual light verb constructions. In Proceedings of the LAW 2010, pages 82–90, Association for Computational Linguistics. find
  33. Jagfeld, Glorianna, and Lonneke van der Plas. 2015. Towards a better semantic role labelling of complex predicates. In Proceedings of NAACL Student Research Workshop, pages 33–39, Denver. find
  34. Jurafsky, Daniel, and James H. Martin. 2009. Speech and Language Processing, 2nd Edition, Prentice-Hall, Inc., Upper Saddle River, NJ. find
  35. Kiela, Douwe, and Stephen Clark. 2014. A systematic study of semantic vector space model parameters. In Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC) at EACL, pages 21–30. Köper, Maximilian, and Sabine Schulte im Walde. 2016. Distinguishing lite… find
  36. Landauer, Thomas K., Peter W. Foltz, and Darrell Laham. 1998. An introduction to latent semantic analysis. Discourse Processes, 25(2-3):259–284. find
  37. Lapesa, Gabriella, and Stefan Evert. 2014. A large scale evaluation of distributional semantic models: Parameters, interactions and model selection. Transactions of the Association for Computational Linguistics, 2:531–545. find
  38. Lapesa, Gabriella, and Stefan Evert. 2017. Large-scale evaluation of dependency-based DSMs: Are they worth the effort? In EACL 2017, pages 394–400. find
  39. Lauer, Mark. 1995. How much is enough?: Data requirements for statistical NLP. CoRR, abs/cmp-lg/9509001. — [arXiv:cmp-lg/9509001](https://arxiv.org/abs/cmp-lg/9509001) find
  40. Levy, Omer, Yoav Goldberg, and Ido Dagan. 2015. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3:211–225.
  41. Lin, Dekang. 1998. Automatic retrieval and clustering of similar words. In Proceedings of the 17th International Conference on Computational Linguistics (Volume 2), pages 768–774.
  42. Lin, Dekang. 1999. Automatic identification of non-compositional phrases. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics on Computational Linguistics, pages 317–324.
  43. McCarthy, Diana, Bill Keller, and John Carroll. 2003. Detecting a continuum of compositionality in phrasal verbs. In Proceedings of the Association for Computational Linguistics 2003 Workshop on Multiword Expressions: Analysis, Acquisition and Treatment, pages 73–80, Association for Computational Li… find
  44. Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111–3119.
  45. Mikolov, Tomas, Wen-tau Yih, and Geoffrey Zweig. 2013. Linguistic regularities in continuous space word representations. In HLT-NAACL, pages 746–751.
  46. Mitchell, Jeff, and Mirella Lapata. 2008. Vector-based models of semantic composition. In Association for Computational Linguistics, pages 236–244. find
  47. Mitchell, Jeff, and Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive Science, 34(8):1388–1429. find
  48. Mohammad, Saif, and Graeme Hirst. 2012. Distributional measures of semantic distance: A survey. CoRR, abs/1203.1858. — [arXiv:1203.1858](https://arxiv.org/abs/1203.1858) find
  49. Nakov, Preslav. 2008. Paraphrasing verbs for noun compound interpretation. In Proceedings of the LREC Workshop Towards a Shared Task for MWEs, pages 46–49. find
  50. Nakov, Preslav. 2013. On the interpretation of noun compounds: Syntax, semantics, and entailment. Natural Language Engineering, 19:291–330. find
  51. Nivre, Joakim, Johan Hall, and Jens Nilsson. 2006. Maltparser: A data-driven parser-generator for dependency parsing. In Proceedings of the Conference on Language Resources and Evaluation (Volume 6), pages 2216–2219. Padó, Sebastian, and Mirella Lapata. 2003. Constructing semantic space models from…
  52. Pennington, Jeffrey, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Association for Computational Linguistics, Doha, Qatar.
  53. Ramisch, Carlos, Silvio Cordeiro, Leonardo Zilio, Marco Idiart, Aline Villavicencio, and Rodrigo Wilkens. 2016. How naked is the naked truth? A multilingual lexicon of nominal compound compositionality. In The 54th Annual Meeting of the Association for Computational Linguistics, pages 156–161. find
  54. Ramisch, Carlos, Silvio Ricardo Cordeiro, and Aline Villavicencio. 2016. Filtering and measuring the intrinsic quality of human compositionality judgments. In Proceedings of the 12th Workshop on Multiword Expressions (MWE 2016), pages 32–37, Berlin. find
  55. Reddy, Siva, Diana McCarthy, and Suresh Manandhar. 2011. An empirical study on compositionality in compound nouns. In Proceedings of the 5th International Joint Conference on Natural Language Processing 2011 (IJCNLP 2011), pages 210–218, Chiang Mai, Thailand. Ren, Zhixiang, Yajuan Lü, Jie Cao, Qun…
  56. Riedl, Martin, and Chris Biemann. 2015. A single word is not enough: Ranking multiword expressions using distributional semantics. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2430–2440, Association for Computational Linguistics. Roller, Stephen, a… find
  57. Sag, Ivan A, Timothy Baldwin, Francis Bond, Ann Copestake, and Dan Flickinger. 2002, Multiword expressions: A pain in the neck for NLP. In Computational Linguistics and Intelligent Text Processing. Springer, New York, pages 1–15.
  58. Salehi, Bahar, Paul Cook, and Timothy Baldwin. 2014. Using distributional similarity of multi-way translations to predict multiword expression compositionality. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 472–481, Gothenburg,… find
  59. Salehi, Bahar, Paul Cook, and Timothy Baldwin. 2015. A word embedding approach to predicting the compositionality of multiword expressions. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 977–983… find
  60. Salehi, Bahar, Nitika Mathur, Paul Cook, and Timothy Baldwin. 2015. The impact of multiword expression compositionality on machine translation evaluation. In Proceedings of the 11th Workshop on Multiword Expressions, pages 54–59, Association for Computational Linguistics, Denver. find
  61. Salle, Alexandre, Aline Villavicencio, and Marco Idiart. 2016. Matrix factorization using window sampling and negative sampling for improved word representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 419–424, Berli… find
  62. Schmid, Helmut. 1995. Treetagger—A language independent part-of-speech tagger. Institut für Maschinelle Sprachverarbeitung, Universität Stuttgart, 43:28. find
  63. Schneider, Nathan, Dirk Hovy, Anders Johannsen, and Marine Carpuat. 2016. find
  64. SemEval 2016 task 10: Detecting minimal semantic units and their meanings (DiMSUM). In Proceedings of SemEval, pages 546–559, San Diego.
  65. Schone, Patrick, and Daniel Jurafsky. 2001. Is knowledge-free induction of multiword unit dictionary headwords a solved problem? In Proceedings of Empirical Methods in Natural Language Processing, pages 100–108, Pittsburgh. find
  66. Schulte im Walde, Sabine, Anna Hätty, Stefan Bott, and Nana Khvtisavrishvili. 2016. GhoSt-NN: A representative gold standard of German noun-noun compound. In Proceedings of the Conference on Language Resources and Evaluation, pages 2285–2292. Schulte im Walde, Sabine, Stefan Müller, and Stefan Rol… find
  67. Socher, Richard, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201–1211…
  68. Stymne, Sara, Nicola Cancedda, and Lars Ahrenberg. 2013. Generation of compound words in statistical machine translation into compounding languages. Computational Linguistics, 39(4):1067–1108. find
  69. Tsvetkov, Yulia, and Shuly Wintner. 2012. Extraction of multi-word expressions from small parallel corpora. Natural Language Engineering, 18(04):549–573. find
  70. Turney, Peter D., and Patrick Pantel. 2010. From frequency to meaning: vector space models of semantics. Journal of Artificial Intelligence Research, 37(1):141–188.
  71. Van de Cruys, Tim, Laura Rimell, Thierry Poibeau, and Anna Korhonen. 2012. Multiway tensor factorization for unsupervised lexical acquisition. In COLING 2012, pages 2703–2720. find
  72. Yazdani, Majid, Meghdad Farahmand, and James Henderson. 2015. Learning semantic composition to detect non-compositionality of multiword expressions. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1733–1742, Association for Computational Linguistics,… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cordeiro-2019-unsupervised-compositionality-prediction-nominal-compounds/