Elamur Research
☰
home / papers / 2018 / Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling

Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P18-2058.pdf

Summary

Most strong neural semantic role labeling (SRL) systems are BIO taggers. They handle one predicate at a time, assume gold predicates are given, and cannot use span-level features. In practice they need a separate, error-prone predicate identification step. He, Lee, Levy and Zettlemoyer propose an end-to-end model that predicts all predicates, their argument spans, and the role labels in a single pass. The model adapts the span-embedding approach of the Lee et al. (2017) coreference system. It builds BiLSTM-based representations for every candidate span and predicate word, then independently classifies each predicate–span pair into a role or "no relation". Beam pruning keeps this computation tractable. The model was evaluated on CoNLL 2005 and CoNLL 2012 (OntoNotes 5.0). Without gold predicates it achieved the best PropBank SRL results reported to date, outperforming the previous top pipeline in every setting. With ELMo it also led the gold-predicate setting. The work matters because it handles the realistic setting where predicates are unknown. It also shows that span-pair models transfer well to other span-relation tasks.

Key points

  • End-to-end gains: the joint model beats the He et al. (2017) pipeline by 1.3–6.0 F1 across settings. The largest gains are on out-of-domain Brown data and on CoNLL 2012, which includes nominal predicates.
  • Best scores with ELMo:
    • End-to-end test F1: 86.0 on WSJ, 76.1 on Brown, 82.9 on CoNLL 2012.
    • Gold-predicate F1: 87.4 on WSJ, 80.4 on Brown, 85.5 on OntoNotes.
  • Efficient pruning: separate argument and predicate beams rank candidates by unary scores, and span width is capped (e.g. W = 30). Keeping 0.4 predicates per word retains over 99.7% of argument-bearing predicates on CoNLL 2012 dev.
  • Strengths: the model handles long-distance arguments better than BIO taggers. Without any explicit syntax, it reaches 95.0% agreement with gold syntactic spans, compared with 94.3% for He et al. (2017).
  • Weakness: because each pair is classified independently, global consistency suffers. Complete-predicate accuracy is 62.6%, versus 64.3–66.4% for BIO systems, and duplicate core roles are more frequent. Constrained decoding removes the duplicates but barely changes F1. Code is available at github.com/luheng/lsgn.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Topics: Semantic role labeling

Cited by

101 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 90 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-2058

@inproceedings{he2018jointly,
  title = {{Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling}},
  author = {He, Luheng and Lee, Kenton and Levy, Omer and Zettlemoyer, Luke},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-2058},
  url = {https://doi.org/10.18653/v1/p18-2058}
}

References

23 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. 5 Punyakanok et al. (2008) described a list of global constraints for SRL systems, e.g., there can be at most one core argument of each type for each predicate. 6 Arguments with labels ARG0,ARG1,. . . ,ARG5 and AA. find
  2. Steven Bird. 2006. Nltk: the natural language toolkit. In ACL. find
  3. Xavier Carreras and Lluı́s Màrquez. 2005. Introduction to the conll-2005 shared task: Semantic role labeling. In CoNLL. find our card
  4. Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, and Xiaodong Shi. 2018. Deep semantic role labeling with self-attention. In AAAI. find
  5. Nicholas FitzGerald, Julian Michael, Luheng He, and Luke Zettlemoyer. 2018. Large-scale qa-srl parsing. In ACL. find our card
  6. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. find
  7. Nicholas FitzGerald, Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. 2015. Semantic role labeling with neural network factors. In EMNLP. find
  8. Bishan Yang and Tom M. Mitchell. 2017. A joint sequential and relational model for frame-semantic parsing. In EMNLP. find
  9. Luheng He, Kenton Lee, Mike Lewis, and Luke S. Zettlemoyer. 2017. Deep semantic role labeling: What works and what’s next. In ACL. find our card
  10. Lingpeng Kong, Chris Dyer, and Noah A Smith. 2016. Segmental recurrent neural networks. In ICLR. find
  11. Yu Zhang, Guoguo Chen, Dong Yu, Kaisheng Yaco, Sanjeev Khudanpur, and James Glass. 2016. Highway long short-term memory rnns for distant speech recognition. In ICASSP. find
  12. Kenton Lee, Luheng He, Mike Lewis, and Luke S. Zettlemoyer. 2017. End-to-end neural coreference resolution. In EMNLP. find our card
  13. Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In ACL. find
  14. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In NAACL. find our card
  15. Diego Marcheggiani, Anton Frolov, and Ivan Titov. 2017. A simple and accurate syntax-agnostic neural model for dependency-based semantic role labeling. In CoNLL. find our card
  16. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using lstms on sequences and tree structures. In ACL. find our card
  17. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL. find
  18. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using ontonotes. In CoNLL. find
  19. Vasin Punyakanok, Dan Roth, Wen tau Yih, Dav Zimak, and Yuancheng Tu. 2004. Semantic role labeling via generalized inference over classifiers. In CoNLL. find
  20. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics. find our card
  21. Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In ACL. find our card
  22. Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. 2018. Linguistically-Informed Self-Attention for Semantic Role Labeling. arXiv preprint. find our card
  23. Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. 2015. Efficient inference and structured learning for semantic role labeling. Transactions of the Association for Computational Linguistics. find