Elamur Research⌕
☰
home / papers / 2021 / Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing

Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing

Paper · 2021 · Shilin Zhou, Qingrong Xia, Zhenghua Li et al.

Summary

Span-based semantic role labeling (SRL) identifies predicates and the multi-word argument spans attached to them. The two dominant approaches are slow. BIO-based sequence labeling re-encodes the sentence once per predicate, and span-based graph parsing faces a cubic search space that requires heuristic pruning. Zhou et al. turn end-to-end span-based SRL into word-based graph parsing, adapting techniques from semantic dependency graph parsing. A pseudo Root node points to every predicate, and argument words are linked to their predicate through position-aware edge labels, an idea taken from Chinese word segmentation and NER tagging. They test four labeling schemata (BES, BE, BIES, BII) on top of the second-order parser of Wang et al. (2019). A constrained Viterbi post-processing step repairs graphs that cannot be decoded into valid SRL structures. On CoNLL05 and CoNLL12, the method generally matches or beats earlier systems in both end-to-end and predicate-given settings, with and without PLMs. It is also much faster. The results suggest that boundary information is the most important signal for finding arguments.

Key points

  • Four schemata were compared. BES links only an argument's first and last words to the predicate and gives single-word arguments a separate S label. It outperformed BE, BIES and BII in every setting. An extra BIS variant performed well below the best schema in preliminary tests.
  • Throughput reaches 669 sentences/s without a PLM and 252 sentences/s with one, avoiding the per-predicate encoding and O(n³) span search of earlier parsers.
  • Evaluation covers CoNLL05 (WSJ and Brown test sets) and CoNLL12. Results are reported with plain BiLSTM, ELMo and BERT encoders, in both end-to-end and predicate-given modes. The predicate-given mode adds a predicate embedding to the input.
  • The model trails the concurrent tree-parsing approach of Zhang et al. (2022) by a small margin. The authors attribute this to tuning, since they fine-tuned BERT for 10 iterations versus 20 in Zhang et al.
  • Code, configuration files and trained models are released at github.com/zsLin177/SRL-as-GP. No external syntactic information is used.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Soochow UniversitySuzhou · China71 works · 4,889 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksArts and Humanities · 17,959 worksLanguage and Linguistics · 17,789 works

🧭 Topics 1Research subjects and languages the work is about.Semantic role labeling · 148 works

Cite

Cite as Shilin Zhou, Qingrong Xia, Zhenghua Li, Yu Zhang, Yu Hong and Min Zhang. 2021. Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing. arXiv:2112.02970.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2112.02970 · arXiv 2112.02970

@misc{zhou2021fast,
  title = {{Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing}},
  author = {Shilin Zhou, Qingrong Xia, Zhenghua Li, Yu Zhang, Yu Hong and Min Zhang},
  year = {2021},
  doi = {10.48550/arxiv.2112.02970},
  eprint = {2112.02970},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2112.02970}
}

References

35 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rexhina Blloshmi, Simone Conia, Rocco Tripodi, and Roberto Navigli. 2021. Generating senses and roles: An end-to-end model for dependency- and span-based semantic role labeling. In Proceedings of IJ-CAI, pages 3786–3793. find
  2. Jiaxun Cai, Shexia He, Zuchao Li, and Hai Zhao. 2018. A full end-to-end semantic role labeler, syntactic-agnostic over syntactic-aware? In Proceedings of ACL, pages 2753–2765. find
  3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171– 4186.
  4. Timothy Dozat and Christopher D. Manning. 2018. Simpler but more accurate semantic dependency parsing. In Proceedings of ACL, pages 484–490.
  5. Jan Hajič, Eva Hajičová, Jarmila Panevová, Petr Sgall, Ondřej Bojar, Silvie Cinková, Eva Fučíková, Marie Mikulová, Petr Pajas, Jan Popelka, Jiří Semecký, Jana Šindlerová, Jan Štěpánek, Josef Toman, Zdeňka Urešová, and Zdeněk Žabokrtský. 2012. Announcing Prague Czech-English Dependency Treeba… find
  6. Luheng He, Kenton Lee, Omer Levy, and Luke Zettlemoyer. 2018. Jointly predicting predicates and arguments in neural semantic role labeling. In Proceedings of ACL, pages 364–369.
  7. Luheng He, Kenton Lee, Mike Lewis, and Luke Zettlemoyer. 2017. Deep semantic role labeling: What works and what’s next. In Proceedings of ACL, pages 473–483.
  8. Angelina Ivanova, Stephan Oepen, Lilja Øvrelid, and Dan Flickinger. 2012. Who did what to whom? A contrastive study of syntacto-semantic dependencies. In Proceedings of LAW, pages 2–11. find
  9. Ishan Jindal, Ranit Aharonov, Siddhartha Brahma, Huaiyu Zhu, and Yunyao Li. 2020. Improved semantic role labeling using parameterized neighborhood memory adaptation. arXiv preprint arXiv:2011.14459. — [arXiv:2011.14459](https://arxiv.org/abs/2011.14459) find
  10. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of NAACL-HLT, pages 260–270.
  11. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of ACL, pages 7871–7880.
  12. Zuchao Li, Shexia He, Hai Zhao, Yiqing Zhang, Zhuosheng Zhang, Xi Zhou, and Xiang Zhou. 2019. Dependency or span, end-to-end uniform semantic role labeling. In Proceedings of AAAI, pages 6730–6737. find
  13. Zuchao Li, Hai Zhao, Rui Wang, and Kevin Parnow. 2020. High-order semantic role labeling. In Findings of EMNLP, pages 1134–1151. find
  14. Ding Liu and Daniel Gildea. 2010. Semantic role features for machine translation. In Proceedings of COLING, pages 716–724. find
  15. Diego Marcheggiani, Jasmijn Bastings, and Ivan Titov. 2018. Exploiting semantics in neural machine translation with graph convolutional networks. In Proceedings of NAACL-HLT, pages 486–492. find
  16. Yusuke Miyao and Jun’ichi Tsujii. 2004. Deep linguistic analysis for the accurate identification of predicate-argument relations. In Proceedings of COLING. find
  17. Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinková, Dan Flickinger, Jan Hajic, and Zdenka Uresova. 2015. find
  18. Semeval 2015 task 18: Broad-coverage semantic dependency parsing. In Proceedings of SemEval, pages 915–926. find
  19. Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Dan Flickinger, Jan Hajič, Angelina Ivanova, and Yi Zhang. 2014. find
  20. SemEval 2014 task 8: Broad-coverage semantic dependency parsing. In Proceedings of SemEval, pages 63–72. find
  21. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational linguistics, 31(1):71–106.
  22. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of EMNLP, pages 1532–1543.
  23. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL-HLT, pages 2227–2237.
  24. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of EMNLP-CoNLL, pages 1–40.
  25. Peng Shi and Jimmy Lin. 2019. Simple bert models for relation extraction and semantic role labeling. arXiv preprint arXiv:1904.05255.
  26. Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. 2018. Linguistically-informed self-attention for semantic role labeling. In Proceedings of EMNLP, pages 5027–5038.
  27. Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, and Xiaodong Shi. 2018. Deep semantic role labeling with self-attention. In Proceedings of AAAI.
  28. Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2015. Machine comprehension with syntax, frames, and semantics. In Proceedings of ACL-IJCNLP, pages 700–706. find
  29. Xinyu Wang, Jingxian Huang, and Kewei Tu. 2019. Second-order semantic dependency parsing with end-to-end neural networks. In Proceedings of ACL, pages 4609–4618. find
  30. Qingrong Xia, Zhenghua Li, Min Zhang, Meishan Zhang, Guohong Fu, Rui Wang, and Luo Si. 2019. Syntax-aware neural semantic role labeling. In Proceedings of AAAI, pages 7305–7313. find
  31. Yu Zhang, Zhenghua Li, and Min Zhang. 2020. Efficient second-order TreeCRF for neural dependency parsing. In Proceedings of ACL, pages 3295–3305.
  32. Yu Zhang, Qingrong Xia, Shilin Zhou, Yong Jiang, Guohong Fu, and Min Zhang. 2022. Semantic role labeling as dependency parsing: Exploring latent tree structures inside arguments. In Proceedings of COLING. find
  33. Zhisong Zhang, Emma Strubell, and Eduard Hovy. 2021. Comparing span extraction methods for semantic role labeling. In Proceedings of SPNLP, pages 67–77. find
  34. Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In Proceedings of ACL-IJCNLP, pages 1127–1137.
  35. Junru Zhou, Zuchao Li, and Hai Zhao. 2020. Parsing all: Syntax and semantics, dependencies and spans. In Findings of EMNLP, pages 4438–4449. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhou-2021-fast-accurate-end-end-span/