Elamur Research⌕
☰
home / papers / 2018 / Syntax for Semantic Role Labeling, To Be, Or Not To Be

Syntax for Semantic Role Labeling, To Be, Or Not To Be

Summary

Some recent neural semantic role labeling (SRL) systems perform very well without any syntactic input. This raises the question of whether dependency parses still help. He et al. examine this for dependency-based SRL.

They split the task into two steps, predicate disambiguation and argument labeling. Each step uses a BiLSTM encoder topped by a deep MLP with highway connections. Before labeling, an extended k-order pruning algorithm selects argument candidates. It climbs from the predicate to the root of the dependency tree and, at each node, gathers descendants up to distance k. A small k keeps only syntactically close words. A very large k effectively passes the whole sentence, so one framework covers both syntax-aware and syntax-agnostic modes.

Each word is represented by:

  • a predicate indicator embedding;
  • CNN+BiLSTM character features;
  • word, lemma, POS and dependency-label embeddings.

The model sets new best scores on CoNLL-2008/2009 for English and Chinese. The authors also use the Sem-F1/LAS ratio to compare SRL systems fairly when their parsers differ in quality. They conclude that syntax still matters, although its advantage has shrunk to roughly 1–2 points.

Key points

  • Evaluated on CoNLL-2009 English and Chinese with the standard splits. Pre-trained embeddings were GloVe for English and Word2Vec trained on Chinese Wikipedia.
  • Architecture details: a 4-layer BiLSTM with hidden size 512 and a 10-layer highway MLP with ReLU. The predicate disambiguation model is identical except that it omits the dependency-relation feature.
  • The Sem-F1/LAS ratio separates SRL quality from parser quality. By this ratio, most post-CoNLL-2009 gains came from better parsers rather than better SRL components.
  • Noisy syntax (LAS 85–100%) was simulated with a random tree generator. Under this noise, tenth-order pruning stayed stable, while first-order pruning lost about 1–5 F1 because it misses true arguments.
  • The authors estimate that roughly 90% LAS input would push Sem-F1 above 90%. They also note that the syntax-aware vs. syntax-agnostic gap has fallen from about 10% to 1–2%.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Semantic role labeling · Syntax & grammar formalisms

Institutions

🎓Shanghai Jiao Tong UniversityShanghai · China43 works · 12 961 citations

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1192

@inproceedings{he2018syntax,
  title = {{Syntax for Semantic Role Labeling, To Be, Or Not To Be}},
  author = {He, Shexia and Li, Zuchao and Zhao, Hai and Bai, Hongxiao},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1192},
  url = {https://doi.org/10.18653/v1/p18-1192}
}

References

35 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). Seattle, Washington, USA, pages 1533–1544. find our card
  2. Anders Björkelund, Bohnet Bernd, Love Hafdell, and Pierre Nugues. 2010. A high-performance syntactic and semantic dependency parser. In Proceedings of the 23rd International Conference on Computational Linguistics (CoLING 2010). Beijing, China, pages 33–36. find
  3. Anders Björkelund, Love Hafdell, and Pierre Nugues. 2009. Multilingual semantic role labeling. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL 2009): Shared Task. Boulder, Colorado, pages 43–48. find our card
  4. Deng Cai, Hai Zhao, Zhisong Zhang, Yuan Xin, Yongjian Wu, and Feiyue Huang. 2017. Fast and accurate neural word segmentation for Chinese. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL). Vancouver, Canada, pages 608–615. find
  5. Ronan Collobert, Jason Weston, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12(1):2493–2537. find our card
  6. Nicholas FitzGerald, Oscar Tckstrm, Kuzman Ganchev, and Dipanjan Das. 2015. Semantic role labeling with neural network factors. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 960–970. find
  7. William Foland and James Martin. 2015. Dependency-based semantic role labeling using convolutional neural networks. In Joint Conference on Lexical and Computational Semantics. pages 279–288. find
  8. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational linguistics 28(3):245–288. find our card
  9. Daniel Gildea and Martha Palmer. 2002. The necessity of parsing for predicate argument recognition. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics (ACL). Philadelphia, Pennsylvania, USA, pages 239–246. find our card
  10. Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martı́, Lluı́s Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, Pavel Straňák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009. The CoNLL-2009 shared task: Syntactic and semantic dependenc… find our card
  11. Luheng He, Kenton Lee, Mike Lewis, and Luke Zettlemoyer. 2017. Deep semantic role labeling: What works and what’s next. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL). Vancouver, Canada, pages 473–483. find our card
  12. Sepp Hochreiter and Jrgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9(8):1735–1780. find
  13. Richard Johansson and Pierre Nugues. 2008. Dependency-based syntactic-semantic analysis with propbank and nombank. In Proceedings of the Twelfth Conference on Computational Natural Language Learning (CoNLL). pages 183–187. find
  14. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR). find
  15. Tao Lei, Yuan Zhang, Lluı́s Màrquez, Alessandro Moschitti, and Regina Barzilay. 2015. High-order low-rank tensors for semantic role labeling. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL: HLT… find
  16. Diego Marcheggiani, Anton Frolov, and Ivan Titov. 2017. A simple and accurate syntax-agnostic neural model for dependency-based semantic role labeling. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017). Vancouver, Canada, pages 411–420. find our card
  17. Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP). Copenhagen, Denmark, pages 1506–1515. find our card
  18. Todor Mihaylov and Anette Frank. 2016. Discourse relation sense classification using cross-argument semantic similarity based on word embeddings. In Proceedings of the Twentieth Conference on Computational Natural Language Learning - Shared Task (CoNLL). Berlin, Germany, pages 100–107. find our card
  19. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS). pages 3111–3119. find our card
  20. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, pages 1532– 1543. find our card
  21. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… find our card
  22. Sameer Pradhan, Wayne Ward, Kadri Hacioglu, James Martin, and Daniel Jurafsky. 2005. Semantic role labeling using different syntactic views. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL). Ann Arbor, Michigan, pages 581–588. 54th Annual Meeting of th… find
  23. Hai Zhao, Wenliang Chen, Jun’ichi Kazama, Kiyotaka Uchimoto, and Kentaro Torisawa. 2009a. Multilingual dependency learning: Exploiting rich features for tagging syntactic and semantic dependencies. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning - Shared Task (… find
  24. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics 34(2):257–287. find our card
  25. Hai Zhao, Wenliang Chen, and Chunyu Kit. 2009b. Semantic dependency parsing of NomBank and Prop-Bank: An efficient integrated approach via a large-scale feature selection. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP). Singapore, pages 30– 39. find
  26. Lianhui Qin, Zhisong Zhang, Hai Zhao, Zhiting Hu, and Eric Xing. 2017. Adversarial connective-exploiting networks for implicit discourse relation classification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL). Vancouver, Canada, pages 1006– 1017. find our card
  27. Hai Zhao, Wenliang Chen, and Guodong Zhou. 2009c. Multilingual dependency learning: A huge feature engineering method to semantic dependency parsing. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning -Shared Task (CoNLL). Boulder, Colorado, pages 55– 60. find
  28. Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany, pages 1192–1202. find our card
  29. Hai Zhao and Chunyu Kit. 2008. Parsing syntactic and semantic dependencies with two single-stage maximum entropy models. In Proceedings of the Twelfth Conference on Computational Natural Language Learning (CoNLL). pages 203–207. find
  30. Chen Shi, Shujie Liu, Shuo Ren, Shi Feng, Mu Li, Ming Zhou, Xu Sun, and Houfeng Wang. 2016. Knowledge-based semantic embedding for machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany, pages 2245–2254. find
  31. Hai Zhao, Xiaotian Zhang, and Chunyu Kit. 2013. Integrative semantic dependency parsing via efficient large-scale feature selection. Journal of Artificial Intelligence Research 46:203–233. find
  32. Rupesh K Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Training very deep networks. In Advances in neural information processing systems. pages 2377–2385. find
  33. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. The conll 2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of the Twelfth Conference on Computational Natural Language Learning - Shared Task (CoNLL). Manchester, England,… find our card
  34. Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. 2016. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). Berlin, Germany, pages 201–206. find our card
  35. Zhisong Zhang, Hai Zhao, and Lianhui Qin. 2016. Probabilistic graph-based dependency parsing with convolutional neural network. In Proceedings of the find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/he-2018-syntax-semantic-role-labeling-be/