Elamur Research⌕
☰
home / papers / 2015 / Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks

Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks

Paper · 2015 · Yubo Chen, Liheng Xu, Kang Liu et al.

Summary

The paper addresses ACE event extraction: finding trigger words, assigning their event subtypes, and attaching arguments with roles. Earlier systems relied on hand-crafted features and on outputs from parsers and other NLP tools. That approach costs a lot of human effort, suffers from sparse one-hot features, and lets upstream errors propagate. The authors replace this pipeline with features learned automatically.

  • Lexical-level clues: Skip-gram word embeddings of the candidate words and their immediate neighbours.
  • Sentence-level clues: a convolutional network with a new dynamic multi-pooling layer. Instead of taking one maximum over the whole sentence, it splits the sentence at the trigger and the candidate argument and max-pools each segment separately.

The resulting model, DMCNN, is applied in two stages: trigger classification, then argument classification. On the ACE2005 dataset it reportedly beats prior state-of-the-art feature-based and structure-based systems. This matters because roughly a quarter of the sentences describe several events, and ordinary max-pooling tends to discard the information needed for the less dominant event.

Key points

  • Multiple-event sentences make up 27.3% of the dataset. This is the main motivation for pooling per segment rather than once per sentence.

  • The task follows the ACE definitions: 8 event types and 33 subtypes, with gold entity labels from ACE used directly, as in most earlier work.

  • The sentence-level input combines three parts:

    • context-word features (CWF);
    • position features (PF), which are learned embeddings of relative distance to the trigger or candidate argument;
    • an event-type feature (EF) for the argument stage.

  • Ablation (Table 5) shows the two feature groups are complementary:

    Features usedTrigger F1Argument F1
    Lexical only64.940.2
    Sentence-level only63.850.7
    Combined69.153.5

  • Many test items are unseen in training with the same label: 34.9% of trigger words and 83.1% of arguments. In the unseen case, embedding-based lexical features beat discrete baseline features by 18.8% for triggers and 8.5% for arguments.

Interesting ideas

  • I1. Pooling split at task-relevant anchors: cutting a sequence at key positions (trigger, argument) and pooling each piece keeps local evidence that a single global max would throw away. This could carry over to relation extraction or any task where a sentence holds several competing signals.
  • I2. Learned relative-distance embeddings: encoding each token's offset from the words being classified tells a CNN which items are in focus without any parser. This is a cheap trick for tasks centred on spans or entities.
  • I3. Two-stage pipeline with shared architecture: detect triggers first, then reuse a similar network, conditioned on the predicted event type, to classify arguments. This keeps the model design uniform while letting the second stage use first-stage output.
  • I4. Evaluating seen vs. unseen items separately: splitting test triggers and arguments by whether they occurred in training with the same label shows where dense features actually help. This is a useful diagnostic for low-resource or sparse-label settings.
  • I5. Ablating lexical vs. sentence features: comparing each feature group alone and combined shows which one drives each subtask. Here sentence context mattered much more for arguments than for triggers.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️Chinese Academy of SciencesBeijing · China68 works · 22,753 citations🔬Institute of AutomationBeijing · China20 works · 13,875 citations

📚 Shelves 5 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Computer Vision and Pattern Recognition · 1,282 works

🧭 Topics 1 Information extraction · 109 works

Appears in

Articles of ours that cite this paper.

Agglomerations Beijing

Cite

Cite as Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng and Jun Zhao. 2015. Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-1017

@inproceedings{chen2015event,
  title = {{Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks}},
  author = {Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng and Jun Zhao},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1017},
  url = {https://doi.org/10.3115/v1/p15-1017}
}

References

25 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ahn. 2006. The stages of event extraction. In Proceedings of ACL, pages 1–8. our card
  2. Marco Baroni, Georgiana Dinu, and Germán Kruszewski. 2014. Dont count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of ACL, pages 238–247. our card
  3. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. The Journal of Machine Learning Research, 3:1137–1155. our card
  4. Chen Chen and V Incent NG. 2012. Joint modeling for chinese event extraction with rich linguistic features. In Proceedings of COLING, pages 529–544. find
  5. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493–2537. our card
  6. Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. 2010. Why does unsupervised pre-training help deep learning? The Journal of Machine Learning Research, 11:625–660. find
  7. Prashant Gupta and Heng Ji. 2009. Predicting unknown time arguments based on cross-event propagation. In Proceedings of ACL-IJCNLP, pages 369– 372. find
  8. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580. find
  9. Yu Hong, Jianfeng Zhang, Bin Ma, Jianmin Yao, Guodong Zhou, and Qiaoming Zhu. 2011. Using cross-entity inference to improve event extraction. In Proceedings of ACL-HLT, pages 1127–1136. our card
  10. Heng Ji and Ralph Grishman. 2008. Refining event extraction through cross-document inference. In Proceedings of ACL, pages 254–262. our card
  11. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. arXiv preprint arXiv:1306.3584. our card
  12. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188. our card
  13. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. our card
  14. Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In Proceedings of AAAI. find
  15. Qi Li, Heng Ji, and Liang Huang. 2013. Joint event extraction via structured prediction with global features. In Proceedings of ACL, pages 73–82. our card
  16. Qi Li, Heng Ji, Yu Hong, and Sujian Li. 2014. Constructing information networks using one single model. In Proceedings of EMNLP, pages 1846– 1851. find
  17. Shasha Liao and Ralph Grishman. 2010. Using document level cross-event inference to improve event extraction. In Proceedings of ACL, pages 789–797. our card
  18. David McClosky, Mihai Surdeanu, and Christopher D Manning. 2011. Event extraction as dependency parsing. In Proceedings of ACL-HLT, pages 1626– 1635. our card
  19. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. our card
  20. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, pages 3111–3119. our card
  21. Frederic Morin and Yoshua Bengio. 2005. Hierarchical probabilistic neural network language model. In Proceedings of AISTATS, pages 246–252. find
  22. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proceedings of ACL, pages 384–394. our card
  23. David Yarowsky. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In Proceedings of ACL, pages 189–196. our card
  24. Matthew D Zeiler. 2012. Adadelta: An adaptive learning rate method. arXiv preprint arXiv:1212.5701. our card
  25. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of COLING, pages 2335–2344. our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chen-2015-event-extraction-dynamic-multi-pooling/