Elamur Research⌕
☰
home / papers / 2015 / Event Detection and Domain Adaptation with Convolutional Neural Networks

Event Detection and Domain Adaptation with Convolutional Neural Networks

Summary

The paper addresses event detection (ED): finding the words that trigger events in text and assigning each one an event type. Strong earlier systems used many hand-built features produced by parsers, name taggers and gazetteers. That approach takes expert effort for every new domain, and errors from those tools pass into the final detector. Nguyen and Grishman instead treat ED as per-token multi-class classification. Each candidate token is seen through a fixed window of surrounding context and fed to a convolutional neural network. Each word is represented by joining a pre-trained word embedding, an embedding of its relative distance to the candidate, and an optional embedding of its entity type. On ACE 2005, the plain CNN matches or beats feature-based systems that rely on external tools. Adding entity types gives state-of-the-art results among systems that use only sentence-level information. In unsupervised domain adaptation, with news as the training domain, the CNN generalizes clearly better than feature-based baselines. This supports learned representations as a way to build extractors that hold up when the text domain changes.

Key points

  • Task and data: ED is framed as a 34-way classification (33 ACE 2005 event subtypes plus "None").
    • The standard split is used: 529 training documents (14,849 sentences), 30 development documents and 40 newswire test documents.
  • Model and settings:
    • Filters of widths {2, 3, 4, 5}, with 150 feature maps per width.
    • A 31-token context window.
    • 50-dimensional position and entity-type embeddings.
    • 300-dimensional word2vec initialization.
    • Dropout of 0.5 and AdaDelta updates; the embedding tables are fine-tuned during training.
  • Ablation on the development set: position embeddings matter most. Without them, F1 drops to roughly 14–16; with them it reaches 68.5, and 70.7 when entity types are added.
  • Two variants: CNN1 uses no external features, while CNN2 adds entity types. CNN2 beats the best sentence-level system of Li et al. (2013b) by about 1.5 F1.
  • Domain adaptation:
    • Setup: train on news (bn+nw), test on bc, cts and wl.
    • Result: CNN2 is best on all three target domains, with significant margins of roughly 5 F1 on bc and cts.

Interesting ideas

  • I1. Encode relative position for token classification. Telling the network how far each context word is from the token being classified is cheap, and here it was the difference between a useless model and a strong one. The trick should transfer to any task that classifies one token in context, such as relations, slot filling or argument roles.
  • I2. Use only what can be learned from raw text. A model without parser or tagger features suffers less when moved to new genres, because it does not inherit the errors those tools make on unfamiliar input. This argues for minimal-preprocessing designs in low-resource or noisy-text settings.
  • I3. Add symbolic information as embeddings. Entity types were fed in as learned vectors using BIO labels on entity heads, rather than as handcrafted features. This keeps the architecture uniform while still benefiting from extra annotation when it exists.
  • I4. Report a realistic evaluation setting. The authors noted that baselines using gold entity annotations are optimistic, and also compared against systems using automatically predicted entities. Matching the evaluation to the information actually available at deployment gives fairer comparisons.
  • I5. Use genre splits as a domain-shift test. Training on one set of ACE genres and testing on held-out genres, with significance testing, is a simple and reproducible robustness check. The same design could be applied to any multi-genre corpus.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

146 papers of our library cite it (of the 140884 whose bibliographies we read); 12 of them have a page here, 134 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-2060

@inproceedings{nguyen2015event,
  title = {{Event Detection and Domain Adaptation with Convolutional Neural Networks}},
  author = {Nguyen, Thien Huu and Grishman, Ralph},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-2060},
  url = {https://doi.org/10.3115/v1/p15-2060}
}

References

32 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain Adaptation with Structural Correspondence Learning. In Proceedings of EMNLP. find our card
  2. Ronan Collobert and Jason Weston. 2008. A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning. In Proceedings of ICML. find
  3. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu and Pavel Kuksa. 2011. Natural Language Processing (Almost) from Scratch. Journal of Machine Learning Research 12:24932537. find our card
  4. Hal Daumé III. 2007. Frustratingly Easy Domain Adaptation. In Proceedings of ACL. find our card
  5. Prashant Gupta and Heng Ji. 2009. Predicting Unknown Time Arguments Based on Cross-Event Propagation. In Proceedings of ACL-IJCNLP. find
  6. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2012. Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors. CoRR, abs/1207.0580. find
  7. Yu Hong, Jianfeng Zhang, Bin Ma, Jian-Min Yao, Guodong Zhou, and Qiaoming Zhu. 2011. Using Cross-entity Inference to Improve Event Extraction. In Proceedings of ACL. find our card
  8. Baotian Hu, Zhengdong Lu, Hang Li, Qingcai Chen. 2014. Convolutional Neural Network Architectures for Matching Natural Language Sentences. In Proceedings of NIPS. find
  9. Ruihong Huang and Ellen Riloff. 2012. Modeling Textual Cohesion for Event Extraction. In Proceedings of AAAI. find
  10. Heng Ji and Ralph Grishman. 2008. Refining Event Extraction through Cross-Document Inference. In Proceedings of ACL. find our card
  11. Nal Kalchbrenner, Edward Grefenstette and Phil Blunsom. 2014. A Convolutional Neural Network for Modeling Sentences. In Proceedings of ACL. find
  12. Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of EMNLP. find our card
  13. Yann LeCun, Léon Bottou, Yoshua Bengio and Patrick Haffner. 1988. Gradient-based Learning Applied to Document Recognition. In Proceedings of the IEEE, 86(11):22782324. find
  14. Peifeng Li, Qiaoming Zhu, and Guodong Zhou. 2013a. Argument Inference from Relevant Event Mentions in Chinese Argument Extraction. In Proceedings of ACL. find
  15. Qi Li, Heng Ji, and Liang Huang. 2013b. Joint Event Extraction via Structured Prediction with Global Features. In Proceedings of ACL. find our card
  16. Shasha Liao and Ralph Grishman. 2010. Using Document Level Cross-event Inference to Improve Event Extraction. In Proceedings of ACL. find our card
  17. Shasha Liao and Ralph Grishman. 2011. Acquiring Topic Features to Improve Event Extraction: in Pre-selected and Balanced Collections. In Proceedings RANLP. find
  18. David McClosky, Eugene Charniak, and Mark Johnson. 2010. Automatic Domain Adaptation for Parsing. In Proceedings of HLT-NAACL. find
  19. David McClosky, Mihai Surdeanu, and Chris Manning. 2011. Event Extraction as Dependency Parsing. In Proceedings of ACL-HLT. find
  20. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Proceedings of NIPS. find
  21. Thien Huu Nguyen and Ralph Grishman. 2014. Employing Word Representations and Regularization for Domain Adaptation of Relation Extraction. In Proceedings of ACL. find
  22. Thien Huu Nguyen and Ralph Grishman. 2015. Relation Extraction: Perspective from Convolutional Neural Networks. In Proceedings of the NAACL Workshop on Vector Space Modeling for NLP (VSM). find our card
  23. Siddharth Patwardhan and Ellen Rilof. 2009. A Unified Model of Phrasal and Sentential Evidence for Information Extraction. In Proceedings of EMNLP. find
  24. Barbara Plank and Alessandro Moschitti. 2013. Embedding Semantic Similarity in Tree Kernels for Domain Adaptation of Relation Extraction. In Proceedings of ACL. find
  25. Sebastian Riedel and Andrew McCallum. 2011. Fast and Robust Joint Models for Biomedical Event Extraction. In Proceedings of EMNLP. find
  26. Sebastian Riedel and Andrew McCallum. 2011. Robust Biomedical Event Extraction with Dual Decomposition and Minimal Domain Adaptation. In Proceedings of the BioNLP Shared Task 2011 Workshop. find
  27. Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng and Grégoire Mesnil. 2014. Learning Semantic Representations Using Convolutional Neural Networks for Web Search. In Proceedings of WWW. find
  28. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word Representations: A Simple and General Method for Semi-supervised Learning. In Proceedings of ACL. find our card
  29. Deepak Venugopal, Chen Chen, Vibhav Gogate and Vincent Ng. 2014. Relieving the Computational Bottleneck: Joint Inference for Event Extraction with High-Dimensional Features. In Proceedings of EMNLP. find
  30. Wen-tau Yih, Xiaodong He, and Christopher Meek. 2014. Semantic Parsing for Single-Relation Question Answering. In Proceedings of ACL. find our card
  31. Matthew D. Zeiler. 2012. ADADELTA: An Adaptive Learning Rate Method. CoRR, abs/1212.5701. find our card
  32. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou and Jun Zhao. 2014. Relation Classification via Convolutional Deep Neural Network. In Proceedings of COLING. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2015-event-detection-domain-adaptation-convolutional/