Elamur Research⌕
☰
home / papers / 2019 / Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning

Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning

Summary

Construction Grammar (CxG) views language as a set of form–meaning pairs that emerge from actual usage. However, the semantics of constructions in CxG is usually described manually, for instance with frames in the spirit of Frame Semantics. Distributional Semantics (DS) derives meaning from corpus data but does not rely on any usage-based theory of grammar. The authors combine these two approaches in a theoretical framework called Distributional Construction Grammar (DisCxG). It has three components. The first is constructions. The second is frames, that is, generalized knowledge about situations and their semantic roles. The third is events with specific participants. All components are described by uniform recursive feature structures modeled on SBCG. Vectors trained on large corpora are added to the semantic level. The work is conceptual and contains no experiments of its own. The authors show how vectors are useful for CxG: they make it possible to compare constructions and frames, recognize idiomatic meaning, model sentence comprehension, and test the theory against psycholinguistic data. The value of the work lies in its proposal to learn the semantics of constructions directly from language usage.

Key points

  • DisCxG consists of three components: Constructions, Frames, Events. The event level is a novelty compared with classical CxG. It was inspired by the cognitive models Generalized Event Knowledge (McRae & Matsuki) and Words-as-Cues (Elman).
  • Semantics has two layers. A frame specifies a prototypical structure of semantic roles, while an event instantiates it with actual participants and the relations between them. Both layers are represented by distributional vectors.
  • The formalism builds on feature structures from SBCG and has a new feature, DS-VECTOR. A vector can be assigned not only to an individual word (for example, read) but also to an abstract syntactic construction.
  • The authors draw on previous results. Vectors of argument constructions, obtained by averaging the vectors of the verbs that typically occur in them (Lebani & Lenci, 2017), reproduced the priming effect from the experiment by Johnson & Goldberg (2013). Busso et al. (2018) found that the Italian Caused-Motion and Dative constructions behave similarly in distributional terms. The sentence comprehension model of Chersoni et al., based on the event graph DEG, accounted for logical metonymy.
  • Future plans include automatically building vector representations of frames and events, creating the first Distributional Construction Treebank, and applying DisCxG in a predictive model of language comprehension.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

Cite as Giulia Rambelli, Emmanuele Chersoni, Philippe Blache, Chu-Ren Huang and Alessandro Lenci. 2019. Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning. Proceedings of the First International Workshop on Designing Meaning Representations.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-3312

@inproceedings{rambelli2019distributional,
  title = {{Distributional Semantics Meets Construction Grammar. towards a Unified Usage-Based Model of Grammar and Meaning}},
  author = {Giulia Rambelli, Emmanuele Chersoni, Philippe Blache, Chu-Ren Huang and Alessandro Lenci},
  booktitle = {Proceedings of the First International Workshop on Designing Meaning Representations},
  year = {2019},
  doi = {10.18653/v1/w19-3312},
  url = {https://doi.org/10.18653/v1/w19-3312}
}

References

63 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. Bybee. 2010. Language, Usage and Cognition. Cambridge University Press. find
  2. Afra Alishahi and Suzanne Stevenson. 2008. A Computational Model of Early Argument Structure Acquisition. Cognitive Science, 32(5):789–834. find
  3. Devin Casenhiser and Adele E Goldberg. 2005. Fast Mapping Between a Phrasal Form and Meaning. Developmental science, 8(6):500–508. find
  4. Irene Amato and Alessandro Lenci. 2017. Story of a Construction: Statistical and Distributional Analysis of the Development of the Italian Gerundival Construction. In G. Marotta and F. S. Lievers, editors, Strutture Linguistiche e Dati Empirici in Diacronia e Sincronia, pages 135–158. Pisa Universit… find
  5. Emmanuele Chersoni, Philippe Blache, and Alessandro Lenci. 2016. Towards a Distributional Model of Semantic Complexity. In Proceedings of the COLING Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), pages 12–22. find
  6. Collin F Baker. 2012. FrameNet, Current Collaborations and Future Goals. Language, Resources and Evaluation, 46(2):269–286. find
  7. Emmanuele Chersoni, Alessandro Lenci, and Philippe Blache. 2017. Logical Metonymy in a Distributional Model of Sentence Comprehension. In Proceedings of *SEM, pages 168–177. find
  8. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In Proceedings of COLING-ACL, pages 86–90, Montreal, Canada. ACL. our card
  9. Emmanuele Chersoni, Enrico Santus, Ludovica Pannitto, Alessandro Lenci, Philippe Blache, and Chu-Ren Huang. 2019. A Structured Distributional Model of Sentence Meaning and Processing. Journal of Natural Language Engineering. To Appear. find
  10. Libby Barak and Adele Goldberg. 2017. Modeling the Partial Productivity of Constructions. In Proceeed-ings of the 2017 AAAI Spring Symposium Series on Computational Construction Grammar and Natural Language Understanding, pages 131–138. find
  11. Silvio Cordeiro, Aline Villavicencio, Marco Idiart, and Carlos Ramisch. 2019. Unsupervised Compositionality Prediction of Nominal Compounds. Computational Linguistics, 45:1–57. find
  12. Marco Baroni, Raffaela Bernardi, and Roberto Zamparelli. 2014. Frege in Space: A Program of Compositional Distributional Semantics. Linguistic Issues in Language Technology, 9(6):5–110. find
  13. Jonathan Dunn. 2017. Computational Learning of Construction Grammars. Language and Cognition, 9(2):254–292. find
  14. Marco Baroni and Alessandro Lenci. 2010. Distributional Memory: A General Framework for Corpus-Based Semantics. Computational Linguistics, 36(4):673–721. our card
  15. Jonathan Dunn. 2019. Frequency vs. Association for Constraint Selection in Usage-Based Construction Grammar. In Proceedings of the NAACL Workshop on Cognitive Modeling and Computational Linguistics. find
  16. Philippe Blache. 2005. Property Grammars: A Fully Constraint-Based Theory. In Skadhauge P. R. Christiansen, H. and J. Villadsen, editors, Constraint Solving and Language Processing. find
  17. CSLP 2004, volume 3438 of LNAI, pages 1–16. Springer, Berlin, Heidelberg. find
  18. Jeffrey L. Elman. 2014. Systematicity in the Lexicon: On Having Your Cake and Eating It Too. In Paco Calvo and John Symons, editors, The Architecture of Cognition, pages 115–145. The MIT Press. find
  19. Philippe Blache. 2016. Representing Syntax by Means of Properties: A Formal Framework for Descriptive Approaches. Journal of Language Modelling, 4(2):183–224. find
  20. Katrin Erk, Sebastian Padó, and Ulrike Padó. 2010. A Flexible, Corpus-Driven Model of Regular and Inverse Selectional Preferences. Computational Linguistics, 36(4):723–763. find
  21. Teresa Botschen, Hatem Mousselly Sergieh, and Iryna Gurevych. 2017. Prediction of Frame-to-Frame Relations in the FrameNet Hierarchy with Frame Embeddings. In Proceedings of the ACL Workshop on Representation Learning for NLP, pages 146–156. find
  22. Charles J. Fillmore. 1982. Frame Semantics. Linguistics in the Morning Calm, pages 111–37. find
  23. Charles J Fillmore. 1988. The Mechanisms of Construction Grammar. Annual Meeting of the Berkeley Linguistics Society, 14:35–55. find
  24. Joyce Tang Boyland. 2009. Usage-Based Models of Language. In D. Eddington, editor, Experimental and Quantitative Linguistics, pages 351–419. Lincom, Munich. find
  25. Adele E. Goldberg. 1999. The Emergence of the Semantics of Argument Structure Constructions. In B. MacWhinney, editor, The Emergence of Language, pages 197–212. Lawrence Erlbaum Publications, Hillsdale, NJ. find
  26. Lucia Busso, Ludovica Pannitto, and Alessandro Lenci. 2018. Modelling Italian Construction Flexibility with Distributional Semantics: Are Constructions Enough? In Proceedings of the Fifth Italian Conference on Computational Linguistics (CLiC-it 2018), pages 68–74. find
  27. Adele E. Goldberg. 2003. Constructions: A New Theoretical Approach to Language. Trends in Cognitive Sciences, 7(5):219–224. find
  28. Alessandro Lenci. 2011. Composing and Updating Verb Argument Expectations: A Distributional Semantic Model. In Proceedings of the ACL Workshop on Cognitive Modeling and Computational Linguistics, pages 58–66. find
  29. Adele E Goldberg. 2006. Constructions at Work: The Nature of Generalization in Language. Oxford University Press on Demand. find
  30. Edward Grefenstette and Mehrnoosh Sadrzadeh. 2015. Concrete Models and Empirical Evaluations for the Categorical Compositional Distributional Model of Meaning. Computational Linguistics, 41(1):71– 118. find
  31. Alessandro Lenci. 2018. Distributional Models of Word Meaning. Annual Review of Linguistics, 4(1):151–171. find
  32. Natalia Levshina and Kris Heylen. 2014. A Radically Data-Driven Construction Grammar: Experiments with Dutch Causative Constructions. In R. Boogaart, T. Colleman, and Rutten G., editors, Extending the Scope of Construction Grammar, pages 17–46. Mouton de Gruyter, Berlin. find
  33. Stefan Th. Gries and Anatol Stefanowitsch. 2004. Extending Collostructional Analysis. A Corpus-Based Perspective on ’Alternations’. International Journal of Corpus Linguistics, 9(1):97–129. find
  34. Peter Hagoort. 2013. MUC (Memory, Unification, Control) and Beyond. Frontiers in Psychology, 4:1– 13. find
  35. Paweł Mandera, Emmanuel Keuleers, and Marc Brysbaert. 2017. Explaining Human Performance in Psycholinguistic Tasks with Models of Semantic Similarity Based on Prediction and Counting: A Review and Empirical Validation. Journal of Memory and Language, 92:57–78. find
  36. Peter Hagoort. 2015. MUC (Memory, Unification, Control): A Model on the Neurobiology of Language Beyond Single Word Processing. In Neurobiology of language, pages 339–347. Elsevier. find
  37. Zellig S. Harris. 1954. Distributional Structure. Word, 10:146–62. find
  38. Brian McElree, Matthew J Traxler, Martin J Pickering, Rachel E Seely, and Ray Jackendoff. 2001. Reading Time Evidence for Enriched Composition. Cognition, 78(1):B17–B25. find
  39. Thomas Hoffman and Graeme Trousdale, editors. 2013. The Oxford Handbook of Construction Grammar. Oxford University Press, Oxford. find
  40. Ken McRae and Kazunaga Matsuki. 2009. People Use their Knowledge of Common Events to Understand Language, and Do So as Quickly as Possible. Language and Linguistics Compass, 3(6):1417–1429. find
  41. Brendan T Johns and Michael N Jones. 2015. Generating Structure from Experience: A Retrieval-Based Model of Language Processing. Canadian Journal of Experimental Psychology, 69(3):233–251. find
  42. Laura A Michaelis. 2013. Sign-Based Construction Grammar. In The Oxford Handbook of Construction Grammar, pages 133–152, Oxford. Oxford University Press. find
  43. Matt A. Johnson and Adele E. Goldberg. 2013. Evidence for Automatic Accessing of Constructional Meaning: Jabberwocky Sentences Prime Associated Verbs. Language and Cognitive Processes, 28(10):1439–1452. find
  44. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems (NIPS 2013), pages 3111–3119. our card
  45. Katharina Kann, Alex Warstadt, Adina Williams, and Samuel R Bowman. 2019. Verb Argument Structure Alternations in Word and Sentence Embeddings. In Proceedings of the Society for Computation in Linguistics (SCiL) 2019, pages 287–297. find
  46. Jeff Mitchell, Mirella Lapata, Vera Demberg, and Frank Keller. 2010. Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure. In Proceedings of ACL, pages 196–206, Uppsala, Sweden. ACL. find
  47. Gina R. Kuperberg and T. Florian Jaeger. 2015. What Do We Mean by Prediction in Language Comprehension? Language Cognition & Neuroscience, 3798. find
  48. Martin Paczynski and Gina R. Kuperberg. 2012. Multiple Influences of Semantic Memory on Sentence Processing: Distinct Effects of Semantic Relatedness on Violations of Real-World Event/State Knowledge and Animacy Selection Restrictions. J Memory and Language, 67(4). find
  49. Gianluca E Lebani and Alessandro Lenci. 2017. Modelling the Meaning of Argument Constructions with Distributional Semantics. In Proceedings of the AAAI 2017 Spring Symposium on Computational Construction Grammar and Natural Language Understanding, pages 197–204. find
  50. Florent Perek. 2016. Using Distributional Semantics to Study Syntactic Productivity in Diachrony: A Case Study. Linguistics, 54(1):149–188. find
  51. Gianluca E. Lebani and Alessandro Lenci. 2018. A Distributional Model of Verb-Specific Semantic Roles Inferences. In Thierry Poibeau and Aline Villavicencio, editors, Language, Cognition, and Computational Models, pages 118–158. Cambridge University Press, Cambridge. find
  52. Florent Perek. 2018. Recent Change in the Productivity and Schematicity of the Way-Construction: A Distributional Semantic Analysis. Corpus Linguistics and Linguistic Theory, 14(1):65–97. find
  53. Josef Ruppenhofer, Michael Ellsworth, Myriam Schwarzer-Petruck, Christopher R Johnson, and Jan Scheffczyk. 2016. FrameNet II: Extended Theory and Practice. find
  54. Ivan A Sag. 2012. Sign-Based Construction Grammar: An Informal Synopsis. In Sign-based construction grammar, volume 193, pages 69–202. CSLI: CSLI Publications. find
  55. Enrico Santus, Emmanuele Chersoni, Alessandro Lenci, and Philippe Blache. 2017. Measuring Thematic Fit with Distributional Feature Overlap. In Proceedings of EMNLP, pages 648–658. find
  56. Asad Sayeed, Stefan Fischer, and Vera Demberg. 2015. Vector-Space Calculation of Semantic Surprisal for Predicting Word Pronunciation Duration. In Proceedings of ACL-IJCNLP, pages 763–773, Beijing, China. ACL. find
  57. Asad Sayeed, Clayton Greenberg, and Vera Demberg. 2016. Thematic Fit Evaluation: An Aspect of Selectional Preferences. In Proceedings of the ACL Workshop on Evaluating Vector-Space Representations for NLP, pages 99–105. find
  58. Marco Silvio Giuseppe Senaldi, Gianluca E Lebani, and Alessandro Lenci. 2016. Lexical Variability and Compositionality: Investigating Idiomaticity with Distributional Semantic Models. In Proceedings of the ACL Workshop on Multiword Expressions, pages 21–31. find
  59. Jennifer Sikos and Sebastian Padó. 2018. Using Embeddings to Compare FrameNet Frames Across Languages. In Proceedings of the COLING Workshop on Linguistic Resources for Natural Language Processing, pages 91–101. find
  60. Michael Tommasello. 2003. Constructing a Language: A Usage-Based Theory of Language Acquisition. Harvard University Press, Cambridge, MA. find
  61. Matthew J Traxler, Martin J Pickering, and Brian McElree. 2002. Coercion in Sentence Processing: Evidence from Eye-Movements and Self-Paced Reading. Journal of Memory and Language, 47(4):530–547. find
  62. Dmitry Ustalov, Alexander Panchenko, Andrei Kutuzov, Chris Biemann, and Simone Paolo Ponzetto. 2018. Unsupervised Semantic Frame Induction Using Triclustering. In Proceedings of ACL, pages 55– 62. find
  63. Kristian Woodsend and Mirella Lapata. 2015. Distributed Representations for Unsupervised Semantic Role Labeling. In Proceedings of EMNLP, pages 2482–2491. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rambelli-2019-distributional-semantics-meets-construction-grammar/