Elamur Research⌕
☰
home / papers / 2015 / Frame-Semantic Role Labeling with Heterogeneous Annotations

Frame-Semantic Role Labeling with Heterogeneous Annotations

Summary

The paper addresses argument identification in frame-semantic parsing: given a predicate and the FrameNet frame it evokes, the system must find the phrases that fill the frame's roles and label them. Fully annotated training data is scarce, with only a few thousand sentences. The authors extend SEMAFOR, an open-source frame-semantic parser for English, with three additional sources of supervision:

  • FrameNet hierarchy features: features copied onto parent roles via Inheritance and Subframe relations, so related roles share evidence.
  • Lexicographic exemplars: FrameNet's large set of partially annotated sentences, added as training data through Daumé-style feature augmentation for domain adaptation.
  • PropBank guide features: predictions from an external PropBank-style SRL system used as extra features.

Training was also made faster by switching to AdaDelta with a squared structured hinge loss. On the FrameNet 1.5 full-text test set, combining these techniques raises argument F1 by about 4 points absolute. With automatically predicted frames, full parsing improves by about 1 point. The work shows that heterogeneous, imperfectly matched resources can compensate for small in-domain corpora in semantic analysis.

Key points

  • Argument identification F1 on the full-text test set:
    • FT baseline: 59.1%
    • Hierarchy features: 60.4%
    • FT+Exemplars with domain adaptation: 62.2%
    • Exemplars as guide features: 60.2%
    • PB-SRL guide features: 59.5%
    • Best combination (FT+Exemplars + PB-SRL guide + hierarchy): 63.1%
  • Data used:
    • FrameNet 1.5 has over 1,000 frames and 12,000 lexical predicates.
    • The full-text portion has about 5,000 sentences.
    • The exemplar corpus has over 140,000 sentences.
    • Role types are connected through 589 Subframe relations and 4,138 Inheritance relations.
  • PropBank guide features came from the Illinois Curator 1.1.4 SRL system.
  • In preliminary experiments, training directly on PropBank data mapped to FrameNet through SemLink lowered performance.
  • Replacing L-BFGS with minibatch AdaDelta cut training time from 729 to 82 minutes, at a cost of only 0.2 F1 (59.3% → 59.1%).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Semantic role labeling

Cited by

28 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 22 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-2036

@inproceedings{kshirsagar2015frame,
  title = {{Frame-Semantic Role Labeling with Heterogeneous Annotations}},
  author = {Kshirsagar, Meghana and Thomson, Sam and Schneider, Nathan and Carbonell, Jaime and Smith, Noah A. and Dyer, Chris},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-2036},
  url = {https://doi.org/10.3115/v1/p15-2036}
}

References

34 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Collin Baker, Michael Ellsworth, and Katrin Erk. 2007. SemEval-2007 Task 19: frame semantic structure extraction. In Proc. of SemEval, pages 99–104. Prague, Czech Republic. find our card
  2. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In Proc. of COLING-ACL, pages 86–90. Montreal, Quebec, Canada. URL http://framenet.icsi.berkeley.edu . ↗ our card
  3. Ulrike Baldewein, Katrin Erk, Sebastian Padó, and Detlef Prescher. 2004. Semantic role labelling with similarity-based generalization using EM-based clustering. In Rada Mihalcea and Phil Edmonds, editors, Proc. of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Sema… find
  4. Claire Bonial, Julia Bonn, Kathryn Conger, Jena D. Hwang, and Martha Palmer. 2014. PropBank: semantics of new predicate types. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proc. o… find our card
  5. Claire Bonial, Kevin Stowe, and Martha Palmer. 2013. Renewing and revising SemLink. In Proc. of the They applied frustratingly easy domain adaptation to learn from FrameNet along with a PropBank-like dataset of nominal frames. 2nd Workshop on Linked Data in Linguistics (LDL-2013): Representing and l… find our card
  6. William W. Cohen and Vitor R. Carvalho. 2005. Stacked sequential learning. In Proc. of IJCAI, pages 671–676. Edinburgh, Scotland, UK. find
  7. Dipanjan Das, Desai Chen, André F. T. Martins, Nathan Schneider, and Noah A. Smith. 2014. Frame-semantic parsing. Computational Linguistics, 40(1):9–56. URL http://www.ark.cs.cmu.edu/SEMAFOR . ↗ our card
  8. Dipanjan Das, André F. T. Martins, and Noah A. Smith. 2012. An exact dual decomposition algorithm for shallow semantic parsing with constraints. In Proc. of *SEM, pages 209–217. Montréal, Canada. find
  9. Dipanjan Das and Noah A. Smith. 2011. Semi-supervised frame-semantic parsing for unknown predicates. In Proc. of ACL-HLT, pages 1435–1444. Portland, Oregon, USA. find our card
  10. Dipanjan Das and Noah A. Smith. 2012. Graph-based lexicon expansion with sparsity-inducing penalties. In Proc. of NAACL-HLT, pages 677–687. Montréal, Canada. find
  11. Hal Daumé, III. 2007. Frustratingly easy domain adaptation. In Proc. of ACL, pages 256–263. Prague, Czech Republic. find
  12. Parvin Sadat Feizabadi and Sebastian Padó. 2015. Combining seemingly incompatible corpora for implicit semantic role labeling. In Proc. of *SEM, pages 40–50. Denver, Colorado, USA. find
  13. Charles J. Fillmore and Collin Baker. 2009. A frames approach to semantic analysis. In Bernd Heine and Heiko Narrog, editors, The Oxford Handbook of Linguistic Analysis, pages 791–816. Oxford University Press, Oxford, UK. find
  14. Michael Fleischman, Namhee Kwon, and Eduard Hovy. 2003. Maximum entropy models for FrameNet classification. In Michael Collins and Mark Steedman, editors, Proc. of EMNLP, pages 49–56. find our card
  15. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288. find our card
  16. Iryna Gurevych, Judith Eckle-Kohler, Silvana Hartmann, Michael Matuschek, Christian M. Meyer, and Christian Wirth. 2012. UBY - a large-scale unified lexical-semantic resource based on LMF. In Proc. of EACL, pages 580–590. Avignon, France. find
  17. Karl Moritz Hermann, Dipanjan Das, Jason Weston, and Kuzman Ganchev. 2014. Semantic frame identification with distributed word representations. In Proc. of ACL, pages 1448–1458. Baltimore, Maryland, USA. find
  18. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: the 90% solution. In Proc. of HLT-NAACL, pages 57–60. New York City, USA. find
  19. Richard Johansson. 2012. Non-atomic classification to improve a semantic role labeler for a low-resource language. In Proc. of *SEM, pages 95–99. Montréal, Canada. find
  20. Richard Johansson. 2013. Training parsers on incompatible treebanks. In Proc. of NAACL-HLT, pages 127–137. Atlanta, Georgia, USA. find
  21. Lingpeng Kong, Nathan Schneider, Swabha Swayamdipta, Archna Bhatia, Chris Dyer, and Noah A. Smith. 2014. A dependency parser for tweets. In Proc. of EMNLP, pages 1001–1012. Doha, Qatar. find our card
  22. Namhee Kwon, Michael Fleischman, and Eduard Hovy. 2004. FrameNet-based semantic parsing using maximum entropy models. In Proc. of Coling, pages 1233–1239. Geneva, Switzerland. find
  23. Ken Litkowski. 2004. SENSEVAL-3 task: Automatic labeling of semantic roles. In Rada Mihalcea and Phil Edmonds, editors, Proc. of SENSEVAL-3, the Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, pages 9– 12. Barcelona, Spain. find
  24. Dong C. Liu and Jorge Nocedal. 1989. On the Limited Memory BFGS Method for Large Scale Optimization. Math. Program., 45(3):503–528. find
  25. André F. T. Martins, Dipanjan Das, Noah A. Smith, and Eric P. Xing. 2008. Stacking dependency parsers. In Proc. of EMNLP, pages 157–166. Honolulu, Hawaii. find
  26. Yuichiroh Matsubayashi, Naoaki Okazaki, and Jun’ichi Tsujii. 2009. A comparative study on generalization of semantic roles in FrameNet. In Proc. of ACL-IJCNLP, pages 19–27. Suntec, Singapore. find our card
  27. Joakim Nivre and Ryan McDonald. 2008. Integrating graph-based and transition-based dependency parsers. In Proc. of ACL-HLT, pages 950–958. Columbus, Ohio, USA. find
  28. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: an annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106. find our card
  29. Ellie Pavlick, Travis Wolfe, Pushpendre Rastogi, Chris Callison-Burch, Mark Drezde, and Benjamin Van Durme. 2015. FrameNet+: Fast paraphrastic tripling of FrameNet. In Proc. of ACL-IJCNLP. Beijing, China. find our card
  30. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics, 34(2):257–287. URL http://cogcomp.cs.illinois.edu/page/software_view/SRL . ↗ our card
  31. Josef Ruppenhofer, Michael Ellsworth, Miriam R. L. Petruck, Christopher R. Johnson, and Jan Scheffczyk. 2010. FrameNet II: extended theory and practice. URL https://framenet2.icsi.berkeley.edu/docs/r1.5/book.pdf . ↗
  32. Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. 2015. Efficient inference and structured learning for semantic role labeling. Transactions of the Association for Computational Linguistics, 3:29–41. find
  33. Cynthia A. Thompson, Roger Levy, and Christopher D. Manning. 2003. A generative model for semantic role labeling. In Machine Learning: ECML 2003, pages 397–408. find
  34. Matthew D. Zeiler. 2012. ADADELTA: An adaptive learning rate method. arXiv:1212.5701 [cs]. URL http://arxiv.org/abs/1212.5701 , arXiv: 1212.5701. ↗ our card

↑ back to top

This page in Markdown →