Elamur Research⌕
☰
home / papers / 2016 / Leveraging FrameNet to Improve Automatic Event Detection

Leveraging FrameNet to Improve Automatic Event Detection

Summary

This paper addresses the shortage of labeled data for event detection (ED) in the ACE 2005 setting, where some event types have only a handful of training examples. The authors observe that FrameNet (FN) frames resemble ACE events: a lexical unit (LU) acts like a trigger, and frame elements act like arguments. They train a simple neural ED model on ACE and use it to produce initial event-type guesses for FN exemplar sentences. These guesses are then refined by joint inference with Probabilistic Soft Logic (PSL). The inference uses soft rules that encourage consistent event types among sentences sharing an LU, a frame, or related frames. Manual and automatic checks show that the detected events are accurate. The authors also derive mappings from frames and LUs to event types. Adding the FN-derived events to the ACE training data raises F1 to a new state of the art. This shows that existing lexical-semantic resources can ease data sparseness in event extraction.

Key points

  • Data: ACE 2005 contains 599 documents with about 6,000 labeled events across 33 event types. FrameNet 1.5 has over 1,000 frames, about 10,000 LUs and roughly 150,000 annotated exemplars.
  • Base model: a three-layer ANN. Its input combines the Skip-gram embedding of the candidate trigger (trained on NYT) with the averaged embedding of the sentence. It is trained with SGD, Adadelta and dropout.
  • Frame relations: PSL inference uses only three of FN's eight frame relations: Inheritance, See also and Perspective on.
  • Results: 3,816 FN events were added to the training data. ANN-FN reaches 70.7 F1, compared with 68.8 for the plain ANN. Recall rises from 60.7 to 65.2, while precision falls only slightly. ANN-FN outperforms Nguyen's CNN (69.0), Chen's DMCNN (69.1) and Liu et al. 2016 (69.4).
  • Mappings: 54 LU-to-event mappings reach ψ = 1.0, and the top frame mappings are mostly correct. All results are released at github.com/subacl/acl16.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Semantic role labeling

Cited by

45 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 43 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-1201

@inproceedings{liu2016leveraging,
  title = {{Leveraging FrameNet to Improve Automatic Event Detection}},
  author = {Liu, Shulin and Chen, Yubo and He, Shizhu and Liu, Kang and Zhao, Jun},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1201},
  url = {https://doi.org/10.18653/v1/p16-1201}
}

References

30 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ahn. 2006. The stages of event extraction. In Proceedings of the Workshop on Annotating and Reasoning about Time and Events, pages 1–8. find
  2. Stephen Bach, Bert Huang, Ben London, and Lise Getoor. 2013. Hinge-loss markov random fields: convex inference for structured prediction. In Proceedings of 29th Annual Meeting of the Association for Uncertainty in Artificial Inteligence, pages 1–10. find
  3. Collin F Baker, Charles J Fillmore, and John B Lowe. 1998. The berkeley framenet project. In Proceedings of 17th Annual Meeting of the Association for Computational Linguistics, pages 86–90. find our card
  4. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. The Journal of Machine Learning Research, 3:1137–1155. find
  5. Aljoscha Burchardt, Marco Pennacchiotti, Stefan Thater, and Manfred Pinkal. 2009. Assessing the impact of frame semantics on textual entailment. Natural Language Engineering, 15(04):527–550. find
  6. Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In Proceedings of 53rd Annual Meeting of the Association for Computational Linguistics, pages 167–176. find
  7. Dipanjan Das, Desai Chen, André FT Martins, Nathan Schneider, and Noah A Smith. 2014. Frame-semantic parsing. Computational Linguistics, 40(1):9–56. find our card
  8. Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. 2010. Why does unsupervised pre-training help deep learning? The Journal of Machine Learning Research, 11:625–660. find
  9. Charles J Fillmore, Christopher R Johnson, and Miriam RL Petruck. 2003. Background to framenet. International Journal of Lexicography, 16(3):235– 250. \#\# Acknowledgements Ana-Maria Giuglea and Alessandro Moschitti. 2006. Shallow semantic parsing based on framenet, verbnet and propbank. European Conf… find
  10. This work was supported by the Natural Science Foundation of China (No. 61533018), the National Basic Research Program of China (No. 2014CB340503) and the National Natural Science Foundation of China (No. 61272332). And this Prashant Gupta and Heng Ji. 2009. Predicting unknown time arguments based o… find
  11. Martin T Hagan, Howard B Demuth, Mark H Beale, et al. 1996. Neural network design. Pws Pub. Boston. find
  12. Srini Narayanan and Sanda Harabagiu. 2004. Question answering based on semantic structures. International Conference on Computational Linguistics. find
  13. Karl Moritz Hermann, Dipanjan Das, Jason Weston, and Kuzman Ganchev. 2014. Semantic frame identification with distributed word representations. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics, pages 1448– 1458. find
  14. Thien Huu Nguyen and Ralph Grishman. 2015. Event detection and domain adaptation with convolutional neural networks. In Proceedings of 53rd Annual Meeting of the Association for Computational Linguistics, pages 365–371. find
  15. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing coadaptation of feature detectors. arXiv preprint arXiv:1207.0580. find
  16. Yu Hong, Jianfeng Zhang, Bin Ma, Jianmin Yao, Guodong Zhou, and Qiaoming Zhu. 2011. Using cross-entity inference to improve event extraction. In Proceedings of 49th Annual Meeting of the Association for Computational Linguistics, pages 1127– 1136. find
  17. Heng Ji and Ralph Grishman. 2008. Refining event extraction through cross-document inference. In Proceedings of 46th Annual Meeting of the Association for Computational Linguistics, pages 254–262. find
  18. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1746–1751. find our card
  19. Angelika Kimmig, Stephen Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. 2012. A short introduction to probabilistic soft logic. In Proceedings of the NIPS Workshop on Probabilistic Programming: Foundations and Applications, pages 1–4. find
  20. Qi Li, Heng Ji, and Liang Huang. 2013. Joint event extraction via structured prediction with global features. In Proceedings of 51st Annual Meeting of the Association for Computational Linguistics, pages 73–82. find
  21. Sebastian Padó and Mirella Lapata. 2005. Cross-linguistic projection of role-semantic information. In Proceedings of the conference on human language technology and empirical methods in natural language processing, pages 859–866. Association for Computational Linguistics. find
  22. Matthew Richardson and Pedro Domingos. 2006. Markov logic networks. Machine learning, pages 107–136. find
  23. Dan Shen and Mirella Lapata. 2007. Using semantic roles to improve question answering. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 12–21. find
  24. Cynthia A Thompson, Roger Levy, and Christopher D Manning. 2003. A generative model for semantic role labeling. European Conference on Machine Learning, pages 397–408. find
  25. David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In Proceedings of 14th Annual Meeting of the Association for Computational Linguistics, pages 189–196. find our card
  26. Matthew D Zeiler. 2012. Adadelta: An adaptive learning rate method. arXiv preprint arXiv:1212.5701. find our card
  27. Shasha Liao and Ralph Grishman. 2010. Using document level cross-event inference to improve event extraction. In Proceedings of 48th Annual Meeting of the Association for Computational Linguistics, pages 789–797. find
  28. Shulin Liu, Kang Liu, Shizhu He, and Jun Zhao. 2016. A probabilistic soft logic based approach to exploiting latent and global information in event classification. In Proceedings of the thirtieth AAAI Conference on Artificail Intelligence. find
  29. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. find
  30. George A. Miller. 1995. Wordnet: a lexical database for english. Communications of the Acm, 38(11):39–41. find our card

↑ back to top

This page in Markdown →