Elamur Research⌕
☰
home / papers / 2014 / Semantic Frame Identification with Distributed Word Representations

Semantic Frame Identification with Distributed Word Representations

Paper · 2014 · Karl Moritz Hermann, Dipanjan Das, Jason Weston et al.

Summary

The paper tackles frame identification: choosing the correct semantic frame for a given predicate in context, the first step of frame-semantic parsing. The authors encode a predicate's syntactic context, taken from automatic dependency parses, by placing generic pre-trained word embeddings into slots reserved for each dependency path type. The concatenated vector is sparse and high-dimensional. Using the WSABIE algorithm with a margin ranking loss, they learn a linear projection of this vector into a low-dimensional space, together with an embedding for every frame label. At prediction time, the frame whose embedding has the highest dot product with the projected context is selected, considering only frames allowed for the lexical unit. Combined with a standard argument identification stage, the system sets a new state of the art on FrameNet frame identification and full parsing. On PropBank SRL, it performs on par with a strong single-parser system. The approach matters because it needs no WordNet or hand-built similarity graphs, so it should transfer more easily to new domains and languages.

Key points

  • The model, called WSABIE EMBEDDING, shares one projection matrix across all frames and examples. It outperforms LOG-LINEAR WORDS, LOG-LINEAR EMBEDDING and the earlier Das et al. (2014) system on FrameNet across all metrics.
  • On FrameNet, even the authors' baselines beat the previous best results. They suggest Das et al. lost accuracy partly through WordNet-based latent smoothing and through normalizing over all frames rather than only the allowed ones.
  • On PropBank, LOG-LINEAR WORDS roughly ties the embedding model: it is slightly better on test and slightly worse on dev. The authors attribute this to PropBank's larger training set and its predicate-specific frame inventory.
  • The model handles rare ambiguous PropBank predicates poorly, because each 512-dimensional label embedding may rest on 10 or fewer examples.
  • SRL was scored with the CoNLL 2005 script. Results match or sometimes exceed the single-parser setting of Punyakanok et al. (2008), and better frame identification generally improved argument scores.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢GoogleMountain View · United States353 works · 364,806 citations🎓University of OxfordOxford · United Kingdom52 works · 154,829 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

Cite

Cite as Karl Moritz Hermann, Dipanjan Das, Jason Weston and Kuzman Ganchev. 2014. Semantic Frame Identification with Distributed Word Representations. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p14-1136

@inproceedings{hermann2014semantic,
  title = {{Semantic Frame Identification with Distributed Word Representations}},
  author = {Karl Moritz Hermann, Dipanjan Das, Jason Weston and Kuzman Ganchev},
  booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2014},
  doi = {10.3115/v1/p14-1136},
  url = {https://doi.org/10.3115/v1/p14-1136}
}

References

32 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. C. F. Baker, C. J. Fillmore, and J. B. Lowe. 1998. The berkeley framenet project. In Proceedings of COLING-ACL. our card
  2. C. Baker, M. Ellsworth, and K. Erk. 2007. SemEval-2007 Task 19: Frame semantic structure extraction. In Proceedings of SemEval. our card
  3. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155. our card
  4. X. Carreras and L. Màrquez. 2004. Introduction to the CoNLL-2004 shared task: Semantic role labeling. In Proceedings of CoNLL. find
  5. X. Carreras and L. Màrquez. 2005. Introduction to the CoNLL-2005 shared task: semantic role labeling. In Proceedings of CoNLL. our card
  6. R. Collobert and J. Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of ICML. find
  7. D. Das, N. Schneider, D. Chen, and N. A. Smith. 2010. Probabilistic frame-semantic parsing. In Proceedings of NAACL-HLT. our card
  8. D. Das, D. Chen, A. F. T. Martins, N. Schneider, and N. A. Smith. 2014. Frame-semantic parsing. Computational Linguistics, 40(1):9–56. our card
  9. M.-C. de Marneffe and C. D. Manning, 2013. Stanford typed dependencies manual. find
  10. C. Fellbaum, editor. 1998. WordNet: an electronic lexical database. find
  11. C. J. Fillmore, C. R. Johnson, and M. R. Petruck. 2003. Background to FrameNet. International Journal of Lexicography, 16(3):235–250. find
  12. C. J. Fillmore. 1982. Frame Semantics. In Linguistics in the Morning Calm, pages 111–137. Hanshin Publishing Co., Seoul, South Korea. find
  13. D. Gildea and D. Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288. our card
  14. E. Hovy, M. Marcus, M. Palmer, L. Ramshaw, and R. Weischedel. 2006. Ontonotes: The 90 In Proceedings of NAACL-HLT. find
  15. R. Johansson and P. Nugues. 2007. LTH: semantic structure extraction using nonprojective dependency trees. In Proceedings of SemEval. find
  16. A. Klementiev, I. Titov, and B. Bhattarai. 2012. Inducing crosslingual distributed representations of words. In Proceedings of COLING. our card
  17. J. Lafferty, A. McCallum, and F. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML. find
  18. D. C. Liu and J. Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(3):503 – 528. find
  19. L. Màrquez, X. Carreras, K. C. Litkowski, and S. Stevenson. 2008. Semantic role labeling: an introduction to the special issue. Computational Linguistics, 34(2):145–159. find
  20. A. Meyers, R. Reeves, C. Macleod, R. Szekely, V. Zielinska, B. Young, and R. Grishman. 2004. The NomBank project: An interim report. In Proceedings of NAACL/HLT Workshop on Frontiers in Corpus Annotation. our card
  21. J. Mitchell and M. Lapata. 2008. Vector-based models of semantic composition. In Proceedings of ACL-HLT. find
  22. M. Palmer, D. Gildea, and P. Kingsbury. 2005. The Proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106. our card
  23. V. Punyakanok, D. Roth, and W. Yih. 2008. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics, 34(2):257–287. our card
  24. R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, and C. D. Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of EMNLP. our card
  25. J. Turian, L. Ratinov, and Y. Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of ACL, Stroudsburg, PA, USA. our card
  26. N. Usunier, D. Buffoni, and P. Gallinari. 2009. Ranking with ordered weighted pairwise classification. In ICML. find
  27. J. Uszkoreit and T. Brants. 2008. Distributed word clustering for large scale class-based language modeling in machine translation. In Proceedings of ACL-HLT. find
  28. K. Q. Weinberger and L. K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research, 10:207–244. find
  29. J. Weston, S. Bengio, and N. Usunier. 2011. Wsabie: Scaling up to large vocabulary image annotation. In Proceedings of IJCAI. find
  30. N. Xue and M. Palmer. 2004. Calibrating features for semantic role labeling. In Proceedings of EMNLP 2004. find
  31. Y. Zhang and J. Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of ACL-HLT. find
  32. Number dev-1 dev-2 dev-3 dev-4 dev-5 dev-6 dev-7 dev-8 dev-9 dev-10 dev-11 dev-12 dev-13 dev-14 dev-15 dev-16 Filename LUCorpus-v0.3 20000420 xin eng-NEW.xml NTI SouthAfrica Introduction.xml LUCorpus-v0.3 CNN AARONBROWN ENG 20051101 215800.partial-NEW.xml LUCorpus-v0.3 AFGP-2002-600045-Trans.xml Pro… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hermann-2014-semantic-frame-identification-distributed-word/