Elamur Research⌕
☰
home / papers / 2017 / A Joint Sequential and Relational Model for Frame-Semantic Parsing

A Joint Sequential and Relational Model for Frame-Semantic Parsing

Summary

The paper tackles frame-semantic parsing in the FrameNet style: given a predicate, the system decides which frame it evokes and then finds its arguments and labels them with that frame's roles. Target identification is left out because consistent labeled data for it is lacking. The authors combine two neural SRL approaches. One is a sequential model, a deep bidirectional LSTM with a CRF layer that tags each word. The other is a relational model, a feed-forward network that classifies candidate argument spans using dependency-based features of the predicate and the argument. They merge the two through knowledge distillation: the relational model is trained to match both the gold labels and the sequential model's output distributions. A separate neural classifier handles frame identification. At test time, frames and roles are decoded jointly in a graphical model with structural constraints, solved with AD³. On FrameNet 1.5 the full system improves markedly on earlier neural and feature-based parsers. This matters because frame structures are a useful input for tasks such as question answering and information extraction.

Key points

  • Results on FrameNet 1.5: full frame-structure extraction reaches 76.6 F1, an absolute gain of 5.7 over the prior state of the art (FitzGerald et al., 2015 variants).
  • Sequential component: an 8-layer DB-LSTM with a CRF layer (following Zhou and Xu, 2015) uses IOB tags, paraphrastic word embeddings, and frame-compatible normalization.
  • Relational component and its gains: it gives higher precision than the sequential model and beats it at every sentence length. Distilling the sequential model into it improves both precision and recall.
  • Results on CoNLL 2005 (PropBank-style SRL): Seq+Rel scores 80.3 on dev, 81.9 on WSJ and 72.0 on Brown. This is comparable to the best published systems, but the gain is smaller than on FrameNet, likely because the argument structures are simpler.
  • Error analysis: the joint model (JointAll) can correct frame-identification mistakes that pipeline systems carry forward, such as reading "train" as Vehicle rather than Education_teaching.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Semantic role labeling

Cited by

25 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 22 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1128

@inproceedings{yang2017joint,
  title = {{A Joint Sequential and Relational Model for Frame-Semantic Parsing}},
  author = {Yang, Bishan and Mitchell, Tom},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1128},
  url = {https://doi.org/10.18653/v1/d17-1128}
}

References

37 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Collin F Baker, Charles J Fillmore, and John B Lowe. 1998. The berkeley framenet project. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics-Volume 1. pages 86–90. find our card
  2. David Belanger, Bishan Yang, and Andrew McCallum. 2017. End-to-end learning for structured prediction energy networks. In ICML. find
  3. Anders Björkelund, Bernd Bohnet, Love Hafdell, and Pierre Nugues. 2010. A high-performance syntactic and semantic dependency parser. In Proceedings of the 23rd International Conference on Computational Linguistics: Demonstrations. Association for Computational Linguistics, pages 33–36. find
  4. Xavier Carreras and Lluı́s Màrquez. 2005. Introduction to the conll-2005 shared task: Semantic role labeling. In CoNLL. find our card
  5. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12(Aug):2493–2537. find our card
  6. Dipanjan Das. 2014. Statistical models for frame-semantic parsing. In Proceedings of Frame Semantics in NLP: A Workshop in Honor of Chuck Fillmore. volume 1929, pages 26–29. find
  7. Dipanjan Das, Desai Chen, André FT Martins, Nathan Schneider, and Noah A Smith. 2014. Frame-semantic parsing. Computational linguistics 40(1):9–56. find our card
  8. Dipanjan Das, Nathan Schneider, Desai Chen, and Noah A Smith. 2010. Probabilistic frame-semantic parsing. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics. Association for Computational Linguistics, pages 948–95… find our card
  9. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In ACL. find our card
  10. Bradley Efron and Robert J Tibshirani. 1994. An introduction to the bootstrap. CRC press. find
  11. Charles J Fillmore, Christopher R Johnson, and Miriam RL Petruck. 2003. Background to framenet. International journal of lexicography 16(3):235– 250. find
  12. Nicholas FitzGerald, Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. 2015. Semantic role labeling with neural network factors. In EMNLP. pages 960–970. find
  13. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational linguistics 28(3):245–288. find our card
  14. Luheng He, Kenton Lee, Mike Lewis, and Luke Zettlemoyer. 2017. Deep semantic role labeling: What works and whats next. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. find our card
  15. Karl Moritz Hermann, Dipanjan Das, Jason Weston, and Kuzman Ganchev. 2014. Semantic frame identification with distributed word representations. In ACL. find
  16. Geoffrey E Hinton. 2002. Training products of experts by minimizing contrastive divergence. Neural computation 14(8):1771–1800. find
  17. Richard Johansson and Pierre Nugues. 2007. Lth: semantic structure extraction using nonprojective dependency trees. In Proceedings of the 4th international workshop on semantic evaluations. Association for Computational Linguistics, pages 227–230. find
  18. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In ICLR. find
  19. Meghana Kshirsagar, Sam Thomson, Nathan Schneider, Jaime Carbonell, Noah A Smith, and Chris Dyer. 2015. Frame-semantic role labeling with heterogeneous annotations. In ACL. find our card
  20. Ding Liu and Daniel Gildea. 2010. Semantic role features for machine translation. In Proceedings of the 23rd International Conference on Computational Linguistics. Association for Computational Linguistics, pages 716–724. find
  21. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc-Closky. 2014. The stanford corenlp natural language processing toolkit. In ACL (System Demonstrations). pages 55–60. find our card
  22. André FT Martins, Mario AT Figeuiredo, Pedro MQ Aguiar, Noah A Smith, and Eric P Xing. 2011. An augmented lagrangian approach to constrained map inference. In ICML. find
  23. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational linguistics 31(1):71– 106. find our card
  24. Mihai Surdeanu, Sanda Harabagiu, John Williams, and Paul Aarseth. 2003. Using predicate-argument structures for information extraction. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1. Association for Computational Linguistics, pages 8–15. find
  25. Mihai Surdeanu, Lluı́s Màrquez, Xavier Carreras, and Pere R Comas. 2007. Combination strategies for semantic role labeling. Journal of Artificial Intelligence Research 29:105–151. find
  26. Swabha Swayamdipta, Miguel Ballesteros, Chris Dyer, and Noah A Smith. 2016. Greedy, joint syntactic-semantic parsing with stack lstms. In CoNLL. find
  27. Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. 2015. Efficient inference and structured learning for semantic role labeling. Transactions of the Association for Computational Linguistics 3:29–41. find
  28. Kristina Toutanova, Aria Haghighi, and Christopher D Manning. 2008. A global joint model for semantic role labeling. Computational Linguistics 34(2):161– 191. find
  29. Jason Weston, Samy Bengio, and Nicolas Usunier. 2011. Wsabie: Scaling up to large vocabulary image annotation. In IJCAI. volume 11, pages 2764–2770. find
  30. John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. Towards universal paraphrastic sentence embeddings. In ICLR. find our card
  31. Nianwen Xue and Martha Palmer. 2004. Calibrating features for semantic role labeling. In EMNLP. pages 88–94. find
  32. Jie Zhou and Wei Xu. 2015. End-to-end learning of semantic role labeling using recurrent neural networks. In ACL (1). pages 1127–1137. find
  33. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics 34(2):257–287. find our card
  34. Michael Roth and Mirella Lapata. 2015. Context-aware frame-semantic role labeling. Transactions of the Association for Computational Linguistics 3:449–460. find our card
  35. Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In ACL. find
  36. Michael Roth and Kristian Woodsend. 2014. Composition of word representations improves semantic role labelling. In EMNLP. Citeseer, pages 407–413. find
  37. Dan Shen and Mirella Lapata. 2007. Using semantic roles to improve question answering. In EMNLP-CoNLL. pages 12–21. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/yang-2017-joint-sequential-relational-model-frame/