Elamur Research⌕
☰
home / papers / 2021 / Joint Universal Syntactic and Semantic Parsing

Joint Universal Syntactic and Semantic Parsing

Summary

Earlier attempts to parse syntax and semantics jointly usually improved one level at the expense of the other. The authors address this with Universal Decompositional Semantics (UDS). Its semantic graphs are derived from the UD parses of the English Web Treebank (EWT) and carry crowdsourced scalar attributes for factuality, genericity, time, word sense and semantic proto-roles. Both annotation layers cover the same sentences, so the effect of the semantic signal can be measured without adding data. The authors extend the transductive sequence-to-graph UDS parser of Stengel-Eskin et al. (2020) so that it also outputs UD trees. They compare several multitask designs with LSTM and new Transformer variants. The best design shares its encoder between the semantic decoder and a biaffine UD parser. This joint system clearly beats the previous best UDS results and reaches state-of-the-art LAS on the semantically valid part of EWT. A UD-focused variant matches the leading UD parser on full EWT. Syntax and semantics help each other even on top of pretrained contextual encoders. Transfer experiments with XLM-R show that the English semantic signal can also improve UD parsing in other languages.

Key points

  • Data: about 20% of EWT is dropped because those sentences lack predicate-argument structure (e.g., greetings, URLs). The cleaned UD subset is released, and UDS attributes are scored on a −3 to 3 scale.
  • Architecture: the best model pairs a shared encoder with a Dozat & Manning biaffine parser. Transformer versions beat LSTM versions on every metric, and fine-tuning the top encoder layers gives the largest extra gain.
  • Multilingual transfer: XLM-R is applied to eight languages drawn from five families: Afrikaans, German, Finnish, French, Galician, Hungarian, Armenian and Kazakh. In the semantics-to-syntax direction, average LAS/UAS rises from 78.56/84.30 to 78.97/84.42 with the encoder-side variant. The intermediate variant lowers it to 77.88/83.43.
  • Per-language results: Finnish loses performance in every variant. Galician, Hungarian and Armenian, which are among the lower-resource languages, gain notably. Kazakh has only 31 training sentences.
  • Code and models: available at github.com/esteng/miso_uds (the PDF extraction shows "miso uds"). The paper was published in TACL, vol. 9, 2021, under a CC-BY 4.0 license.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00396

@article{stengeleskin2021joint,
  title = {{Joint Universal Syntactic and Semantic Parsing}},
  author = {Stengel-Eskin, Elias and Murray, Kenton and Zhang, Sheng and White, Aaron Steven and Durme, Benjamin Van},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00396},
  url = {https://doi.org/10.1162/tacl_a_00396}
}

References

65 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Omri Abend, Tom Kwiatkowski, Nathaniel J. Smith, Sharon Goldwater, and Mark Steedman. 2017. Bootstrapping language acquisition. Cognition, 164:116–143. https://doi.org/10.1016/j.cognition.2017.02.009 — [doi:10.1016/j.cognition.2017.02.009](https://doi.org/10.1016/j.cognition.2017.02.009) ↗
  2. Rich Caruana. 1997. Multitask learning. Machine Learning, 28(1):41–75. https://doi.org/10.1023/A:1007379606734 — [doi:10.1023/a:1007379606734](https://doi.org/10.1023/a:1007379606734) ↗
  3. Omri Abend and Ari Rappoport. 2013. Universal Conceptual Cognitive Annotation (UCCA). In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 228–238, Sofia, Bulgaria. Association for Computational Linguistics. find our card
  4. Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 740–750, Doha, Qatar. Association for Computational Linguistics. https://doi.org/10.3115/v1… ↗ our card
  5. Yoav Artzi, Kenton Lee, and Luke Zettlemoyer. 2015. Broad-coverage CCG semantic parsing with AMR. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1699–1710, Lisbon, Portugal. Association for Computational Linguistics. find
  6. Carol Chomsky. 1969. The Acquisition of Syntax in Children from 5 to 10. Cambridge, MA. MIT Press. find
  7. Yoeng-Jin Chu. 1965. On the shortest arborescence of a directed graph. Scientia Sinica, 14:1396–1400. find
  8. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract Meaning Representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discou… find our card
  9. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association f… find
  10. James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. The Journal of Machine Learning Research, 13(1):281–305. find
  11. Richard F. Cromer. 1970. Children are nice to understand: Surface structure clues for the recovery of a deep structure. British Journal of Psychology, 61(3):397–408. https://doi.org/10.1111/j.2044-8295.1970 .tb01259.x Sebastian Beschke. 2019. Exploring graph-algebraic CCG combinators for syntactic-… — [doi:10.1111/j.2044-8295.1970](https://doi.org/10.1111/j.2044-8295.1970) ↗
  12. Jacob Devlin. 2018. README document. find
  13. Multilingual BERT Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… find
  14. Goran Glavaš and Ivan Vulić. 2020. Is supervised syntactic parsing beneficial for language understanding? an empirical investigation. arXiv preprint arXiv:2008.06788. — [arXiv:2008.06788](https://arxiv.org/abs/2008.06788) find
  15. Lila Gleitman. 1990. The structural sources of verb meanings. Language Acquisition, 1(1):3–55. https://doi.org/10.1207/s15327817la0101 2 ↗
  16. David Dowty. 1991. Thematic proto-roles and argument selection. Language, 67(3):547–619. find
  17. Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martı́, Lluı́s Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, Pavel Straňák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL-2009 shared task: Syntactic and semantic dependenc… find our card
  18. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings. OpenReview.net. https://doi.org/10.1353/lan.1991 .0021 ↗ our card
  19. Jack Edmonds. 1967. Optimum branchings. Journal of Research of the National Bureau of Standards B, 71(4):233–240. https://doi.org/10.6028/jres.071B.032 — [doi:10.6028/jres.071b.032](https://doi.org/10.6028/jres.071b.032) ↗
  20. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733–2743, Hong Kong, Chi… find
  21. Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, N… find our card
  22. Ganesh Jawahar, Benoı̂t Sagot, and Djamé Seddah. 2019. What does BERT learn about the structure of language? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3651–3657, Florence, Italy. Association for Computational Linguistics. https://doi.org/10.18… find our card
  23. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. In Proceedings of Workshop for NLP Open Source Software (NLP-OSS), pages 1–6, Melbourne,… find our card
  24. Association Richard Johansson and Pierre Nugues. 2008. Dependency-based syntactic–semantic analysis with PropBank and NomBank. In CoNLL 2008: Proceedings of the Twelfth Conference on Computational Natural Language Learning, pages 183–187, Manchester, England. find
  25. Coling 2008 Organizing Committee. https://doi.org/10.3115/1596324 .1596355 normalization of self-attention. arXiv preprint arXiv:1910.05895. ↗
  26. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing Universal Dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–279… find our card
  27. Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinková, Dan Flickinger, Jan Hajič, Angelina Ivanova, and Zdeňka Urešová. 2016. Towards comparability of linguistic graph Banks for semantic parsing. In Proceedings of the Tenth International Conference on Language Resources and… find
  28. Jayant Krishnamurthy and Tom M. Mitchell. 2014. Joint syntactic and semantic parsing with Combinatory Categorial Grammar. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1188–1198, Baltimore, Maryland. Association for Computat… find
  29. Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Dan Flickinger, Jan Hajič, Angelina Ivanova, and Yi Zhang. 2014a. find
  30. SemEval 2014 task 8: Broad-coverage semantic dependency parsing. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 63–72, Dublin, Ireland. Association for Computational Linguistics. https://doi.org/10.3115/v1/S14-2008 ↗
  31. Barbara Landau and Lila R. Gleitman. 1985. Language and Experience: Evidence from the Blind Child, Cognitive Science Series, 8, Cambridge, MA. Harvard University Press. find
  32. S. Pinker. 1984. Language Learnability and Language Development. Cambridge, MA. Harvard University Press. find
  33. Steven Pinker. 1979. Formal models of language learning. Cognition, 7(3):217–283. https://doi.org/10.1016/0010-0277(79)90001-5 — [doi:10.1016/0010-0277(79)90001-5](https://doi.org/10.1016/0010-0277%2879%2990001-5) ↗
  34. Mike Lewis, Luheng He, and Luke Zettlemoyer. 2015. Joint A* CCG parsing and semantic role labelling. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1444–1454, Lisbon, Portugal. Association for Computational Linguistics. find
  35. Carl Pollard and Ivan A. Sag. 1994. Head-driven Phrase Structure Grammar, University of Chicago Press. find
  36. Tal Schuster, Ori Ram, Regina Barzilay, and Amir Globerson. 2019. Cross-lingual alignment of contextual word embeddings, with applications to zero-shot dependency parsing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… find
  37. Dipendra Kumar Misra and Yoav Artzi. 2016. Neural shift-reduce CCG semantic parsing. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1775–1786, Austin, Texas. Association for Computational Linguistics. find
  38. Richard Montague. 1970. Universal grammar. Theoria, 36(3):373–398. find
  39. Letitia Naigles. 1990. Children use syntax to learn verb meanings. Journal of Child Language, 17(2):357–374. https://doi.org/10.1017/S0305000900013817 — [doi:10.1017/s0305000900013817](https://doi.org/10.1017/s0305000900013817) ↗
  40. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of Toan Q. Nguyen and Julian Salazar. 2019. Transformers without tears: Improving the the Association for Computational Linguistics… find
  41. Shallow syntax in deep water. arXiv preprint arXiv:1908.11047. — [arXiv:1908.11047](https://arxiv.org/abs/1908.11047) find
  42. Swabha Swayamdipta, Sam Thomson, Chris Dyer, and Noah A. Smith. 2017. Frame-semantic parsing with softmax-margin segmental RNNs and a syntactic scaffold. arXiv preprint arXiv:1706.09528. find our card
  43. Mark Steedman. 2000. The Syntactic Process, 24, MIT Press, Cambridge, MA. https://doi.org/10.7551/mitpress/6591.001 .0001 — [doi:10.7551/mitpress/6591.001](https://doi.org/10.7551/mitpress/6591.001) ↗
  44. Swabha Swayamdipta, Sam Thomson, Kenton Lee, Luke Zettlemoyer, Chris Dyer, and Noah A. Smith. 2018. Syntactic scaffolds for semantic structures. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3772–3782, Brussels, Belgium. Association for Computationa… find
  45. Elias Stengel-Eskin, Aaron Steven White, Sheng Zhang, and Benjamin Van Durme. 2020. Universal decompositional semantic parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8427–8439, Online. Association for Computational Linguistics. https://doi… find
  46. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593–4601, Florence, Italy. Association for Computational Linguistics. https://doi.org/10.18653/v1/P19 -14… ↗ our card
  47. Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. 2018. Linguistically-informed self-attention for semantic role labeling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5027–5038, Brussels, Belgium. Association for Comput… find our card
  48. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. find
  49. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. find
  50. The CoNLL 2008 shared task on joint parsing of syntactic and semantic dependencies. In CoNLL 2008: Proceedings of the Twelfth Conference on Computational Natural Language Learning, pages 159–177, Manchester, England. find our card
  51. Coling 2008 Organizing Committee. https://doi.org/10.3115/1596324 .1596352 — [doi:10.3115/1596324](https://doi.org/10.3115/1596324) ↗
  52. Aaron Steven White. 2020. Lexically triggered veridicality inferences. Handbook of Pragmatics: 22nd Annual Installment, 22:115. https://doi.org/10.1075/hop.22 .lex4 — [doi:10.1075/hop.22](https://doi.org/10.1075/hop.22) ↗
  53. Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2016. Universal decompositional semantics on Universal Dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages… find our card
  54. Swabha Swayamdipta, Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2016. Greedy, joint syntactic-semantic parsing with stack LSTMs. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 187–197, Berlin, Germany. Association for Computational Linguistics.… find
  55. Aaron Steven White, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2018. Lexicosyntactic inference in neural models. find
  56. Swabha Swayamdipta, Matthew Peters, Brendan Roof, Chris Dyer, and Noah A. Smith. 2019. find
  57. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4717–4724, Brussels, Belgium. Association for Computational Linguistics. of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–19, Vancouver, Canada. Associ… — [doi:10.18653/v1/k17](https://doi.org/10.18653/v1/k17) find
  58. Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Subrahmanyan Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2020. The universal decompositional semantics dataset and decomp to… find
  59. Sheng Zhang, Xutai Ma, Kevin Duh, and Benjamin Van Durme. 2019a. AMR parsing as sequence-to-graph transduction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 80–94, Florence, Italy. Association for Computational Linguistics. https://doi.org/10.186… find
  60. Sheng Zhang, Xutai Ma, Kevin Duh, and Benjamin Van Durme. 2019b. Broad-coverage semantic parsing as transduction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages… find
  61. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. find
  62. CoNLL 2018 shared task: Multilingual parsing from raw text to Universal Dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21, Brussels, Belgium. Association for Computational Linguistics. find our card
  63. Sheng Zhang, Rachel Rudinger, and Benjamin Van Durme. 2017. An evaluation of PredPatt and open IE via stage 1 semantic role labeling. In IWCS 2017 — 12th International Conference on Computational Semantics — Short papers. find
  64. Daniel Zeman, Martin Popel, Milan Straka, Jan Hajič, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gokirmak, Anna Nedoluzhko, Silvie Cinková, Jan Hajič Jr., Jaroslava Hlaváčová, Václava Kettnerová, Zdeňka Ur… find
  65. CoNLL 2017 shared task: Multilingual parsing from raw text to Universal Dependencies. In Proceedings Qiji Zhou, Yue Zhang, Donghong Ji, and Hao Tang. 2020. AMR parsing with latent structural information. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, page… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/eskin-2021-joint-universal-syntactic-semantic-parsing/