Elamur Research⌕
☰
home / papers / 2025 / You Shall Know a Construction by the Company it Keeps: Computational Construction Grammar with Embeddings

You Shall Know a Construction by the Company it Keeps: Computational Construction Grammar with Embeddings

Summary

The authors seek a way to combine symbolic computational construction grammar with numerical (distributional) representations. The goal is to preserve the transparency of the symbolic approach while making it more robust to deviations from the training data. The basis is a grammar in the Fluid Construction Grammar (FCG) framework, automatically learned from English corpora with PropBank annotation. It contains 21,052 constructions and builds semantic frames with argument roles for sentences. Vectors are added at three levels. Lemmas receive GloVe embeddings. Construction slots receive prototype vectors of their typical fillers. The similarity of grammatical categories is derived from the slots in which they occur. The FCG unification algorithm is modified: instead of checking for an exact match of symbols, it computes cosine similarity and ranks the options for applying constructions. Examples from the test data show that such a grammar handles unknown verbs. It is also able to choose between competing constructions and to establish links between categories that were not seen during training. The work is important as an argument in favor of a hybrid approach, although for now it is only a prototype.

Key points

  • The base grammar (21,052 constructions) was learned by the fcg-propbank subsystem on the training portions of OntoNotes and the English Web Treebank (EWT). The examples are taken from the test portions of these corpora. Sentences are parsed into constituents by the Berkeley neural parser, and as output the grammar produces PropBank rolesets with the roles arg0/arg1/arg2.
  • First experiment: lemma features are replaced with references to 100-dimensional GloVe vectors. The verb "enrage" was absent from the training data, but it was recognized by the construction for "infuriate" (cosine similarity 0.84). As a result, the frame infuriate.01 with the correct arguments was obtained.
  • Second experiment: during training, construction slots accumulate vectors of the words and phrases that occurred in them. During parsing, the aggregate similarity between slots and their fillers helps choose between competing constructions.
  • The third experiment does without external embeddings: category similarity is computed from their distribution across slots. New links in the category network are created during parsing. For example, "googling" could be integrated into the imperative transitive construction thanks to the proximity of cat-10 and cat-21.
  • There is no quantitative evaluation yet, only a qualitative demonstration on examples. The authors leave scaling up the prototype and a full evaluation for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

Cite as Lara Verheyen, Jonas Doumen, Paul Van Eecke and Katrien Beuls. 2025. You Shall Know a Construction by the Company it Keeps: Computational Construction Grammar with Embeddings. Proceedings of the Second International Workshop on Construction Grammars and NLP.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{verheyen2025you,
  title = {{You Shall Know a Construction by the Company it Keeps: Computational Construction Grammar with Embeddings}},
  author = {Lara Verheyen, Jonas Doumen, Paul {Van Eecke} and Katrien Beuls},
  booktitle = {Proceedings of the Second International Workshop on Construction Grammars and NLP},
  year = {2025},
  url = {https://aclanthology.org/2025.cxgsnlp-1.8/}
}

References

26 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Benjamin K. Bergen and Nancy Chang. 2005. Embodied Construction Grammar in simulation-based language understanding. In Mirjam Fried and Jan-Ola Östman, editors, Construction Grammars: Cognitive Grounding and Theoretical Extensions, pages 147–190. John Benjamins, Amsterdam, Netherlands. find
  2. Alessandro Lenci. 2018. Distributional models of word meaning. Annual review of Linguistics, 4(1):151– 171. find
  3. Natalia Levshina and Kris Heylen. 2014. A radically data-driven construction grammar: Experiments with dutch causative constructions. In Ronny Boogaart, Timothy Colleman, and Gijsbert Rutten, editors, Extending the Scope of Construction Grammar, pages 17–46. De Gruyter Mouton, Berlin, Germany. find
  4. Katrien Beuls and Paul Van Eecke. 2023. Fluid Construction Grammar: State of the art and future outlook. In Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023), pages 41–50. Association for Computational Linguistics. our card
  5. Laura A. Michaelis. 2008. Entity and event coercion in a symbolic theory of syntax. In Jan-Ola Östman and Mirjam Fried, editors, Construction grammars: Cognitive grounding and theoretical extensions, pages 45–88. John Benjamins Publishing Company. find
  6. Ann Bies, Justin Mott, Colin Warner, and Seth Kulick. 2012. English web treebank ldc2012t13. Philadelphia, Linguistic Data Consortium. find
  7. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26 (NIPS 2013), pages 1–9, Red Hook. our card
  8. Philippe Blache, Emmanuele Chersoni, Giulia Rambelli, and Alessandro Lenci. 2024. Composing or Not Composing? Towards Distributional Construction Grammars. arXiv preprint arXiv:2412.07419. — [arXiv:2412.07419](https://arxiv.org/abs/2412.07419) find
  9. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71– 106. our card
  10. Paul Van Eecke. 2018. Generalisation and specialisation operators for computational construction grammar and their application in evolutionary linguistics Research. Ph.D. thesis, Vrije Universiteit Brussel, Brussels: VUB Press. find
  11. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Association for Computational Linguistics. our card
  12. Paul Van Eecke and Katrien Beuls. 2025. PyFCG: Fluid Construction Grammar in Python. arXiv preprint arXiv:2505.12920. find
  13. Remi van Trijp, Katrien Beuls, and Paul Van Eecke. 2022. The FCG Editor: An innovative environment for engineering computational construction grammars. PLOS ONE, 17(6):e0269708. find
  14. Florent Perek. 2016. Using distributional semantics to study syntactic productivity in diachrony: A case study. Linguistics, 54(1):149–188. find
  15. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 6000–6010, Long Beach. our card
  16. Giulia Rambelli, Emmanuele Chersoni, Philippe Blache, Chu-Ren Huang, and Alessandro Lenci. 2019. Distributional semantics meets Construction Grammar. Towards a unified usage-based model of grammar and meaning. In First international workshop on designing meaning representations (DMR 2019). our card
  17. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2013. Ontonotes release 5.0 ldc2013t19. Philadelphia, Linguistic Data Consortium. find
  18. Ivan A. Sag. 2012. Sign-based construction grammar: An informal synopsis. In Hans C. Boas and Ivan A. Sag, editors, Sign-based construction grammar, pages 69–202. CSLI Publications/Center for the Study of Language and Information, Stanford, CA, USA. find
  19. Leonie Weissweiler, Taiqi He, Naoki Otani, David R. Mortensen, Lori Levin, and Hinrich Schütze. 2023. Construction grammar provides unique insight into neural language models. In Proceedings of the First International Workshop on Construction Grammars and NLP (CxGs+NLP, GURT/SyntaxFest 2023), pages… our card
  20. Luc Steels. 2004. Constructivist development of grounded construction grammar. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 9–16. Association for Computational Linguistics. our card
  21. Luc Steels and Joachim De Beule. 2006. Unify and merge in Fluid Construction Grammar. In Symbol Grounding and Beyond, Third International Workshop on the Emergence and Evolution of Linguistic Communication, EELC 2006, Rome, Italy, September 30 - October 1, 2006, Proceedings, volume 4211 of Lecture N… find
  22. Leonie Weissweiler, Valentin Hofmann, Abdullatif Köksal, and Hinrich Schütze. 2022. The better your syntax, the better your semantics? Probing pretrained language models for the English comparative correlative. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processi… find
  23. Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 818–827. our card
  24. Shijia Zhou, Leonie Weissweiler, Taiqi He, Hinrich Schütze, David R. Mortensen, and Lori Levin. 2024. Constructions are so difficult that even large language models get them right for the wrong reasons. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language… find
  25. Harish Tayyar Madabushi, Laurence Romain, Dagmar Divjak, and Petar Milin. 2020. CxGBERT: BERT meets construction grammar. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4020–4032. International Committee on Computational Linguistics. our card
  26. Harish Tayyar Madabushi, Laurence Romain, Petar Milin, and Dagmar Divjak. 2025. Construction grammar and language models. In Mirjam Fried and Kiki Nikiforidou, editors, The Cambridge Handbook of Construction Grammar, pages 572–595. Cambridge University Press, Cambridge, United Kingdom. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/verheyen-2025-you-shall-know-construction-company/