Elamur Research⌕
☰
home / papers / 2016 / A Language-Independent Neural Network for Event Detection

A Language-Independent Neural Network for Event Detection

Summary

The paper addresses event detection: finding the trigger word that signals an event in a sentence and assigning it an event type. Earlier systems relied on hand-built lexical and syntactic features from tools such as parsers and name taggers, which many languages lack. The authors propose HNN, a hybrid neural network that needs no manually engineered features. A Bi-LSTM reads the whole sentence in both directions around a trigger candidate to capture sequence context. A CNN with bigram and trigram filters and a relative-position feature captures chunk-like local patterns. Both outputs are concatenated and passed to a softmax classifier. Inputs are Skip-Gram word embeddings.

The model was tested on ACE2005 for English and Chinese and on ERE for Spanish. On English it reached 73.4% F for trigger classification, beating feature-based and earlier neural baselines. On Chinese it was roughly on par with a feature-rich system, and on Spanish it outperformed LSTM and Bi-LSTM. The work shows that a single feature-free architecture can be moved across languages with minimal adaptation.

Key points

  • English (ACE2005): HNN scored 75.9 F for trigger identification and 73.4 F for classification (P 84.6, R 64.9).
    • Bi-LSTM alone scored 70.6 F on classification.
    • The CNN of Chen et al. (2015) scored 69.1 F, and the Joint Model scored 67.5 F.
  • Chinese (ACE2005): HNN scored 68.2 F for identification, better than Rich-C (66.7) and MaxEnt (60.6).
    • For classification it scored 63.0 F, essentially tied with Rich-C (63.2).
    • Word segmentation was done with the Language Technology Platform.
  • Spanish (ERE): HNN scored about 71.6 F for identification and 55.5 F for classification. Bi-LSTM scored about 68.7 and 50.1.
  • Data sizes (train/dev/test documents):
    • English: 529/30/40
    • Chinese: 513/60/60
    • Spanish: 93/12/12
  • Setup:
    • Embeddings: 300-dimensional Skip-Gram vectors, trained on NYT for English and Gigaword for Chinese and Spanish.
    • CNN: filter widths 2 and 3, 300 feature maps, 5-dimensional position feature.
    • Training: SGD (learning rate 0.03) for English and Adadelta for the other two languages.

Interesting ideas

  • I1. Combining two context encoders. The model joins a sentence-wide bidirectional recurrent encoder with a short-window convolutional encoder. The first captures long-range cues, such as a word in another clause, and the second captures phrase-like chunks. This pairing could help other token-level tasks in low-resource languages where parsers and chunkers are missing.
  • I2. Encoding each candidate separately. A small embedding of each word's relative distance to the candidate token is added to the CNN input. As a result, the same sentence is encoded differently for each candidate. This trick carries over to any task that classifies a specific position or span.
  • I3. Learning chunks instead of computing them. Bigram and trigram convolution filters act as a cheap, learned stand-in for chunking. This could be useful when real chunkers are unavailable or unreliable.
  • I4. An ablation ladder for evaluation. The authors compare RNN, then LSTM, then Bi-LSTM, then the hybrid, which isolates how much each piece adds. For example, the gain from reading the context to the right of the trigger becomes visible. The same design is a cheap way to justify architectural choices in new projects.
  • I5. Testing portability with the same architecture. One unchanged architecture is applied to three languages, with only the embeddings, the optimizer and, for Chinese, word segmentation changed. This is a practical template for checking whether a model truly transfers to languages such as Ukrainian.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

79 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 73 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-2011

@inproceedings{feng2016language,
  title = {{A Language-Independent Neural Network for Event Detection}},
  author = {Feng, Xiaocheng and Huang, Lifu and Tang, Duyu and Ji, Heng and Qin, Bing and Liu, Ting},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-2011},
  url = {https://doi.org/10.18653/v1/p16-2011}
}

References

23 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yang Liu, Furu Wei, Sujian Li, Heng Ji, Ming Zhou, and Houfeng Wang. 2015. A dependency-based neural network for relation classification. arXiv preprint arXiv:1507.04646. find
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. find our card
  3. Fan Miao and Ralph Grishman. 2015. Improving event detection with active learning. In EMNLP. find
  4. Chen Chen, V Incent Ng, and et al. 2012. Joint modeling for chinese event extraction with rich linguistic features. In In COLING. Citeseer. find
  5. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In IN-TERSPEECH, volume 2, page 3. find
  6. Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proces… find our card
  7. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119. find
  8. Thien Huu Nguyen and Ralph Grishman. 2015. Event detection and domain adaptation with convolutional neural networks. Volume 2: Short Papers, page 365. find our card
  9. Yu Hong, Jianfeng Zhang, Bin Ma, Jianmin Yao, Guodong Zhou, and Qiaoming Zhu. 2011. Using cross-entity inference to improve event extraction. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 1127– 1136. Associatio… find our card
  10. Mike Schuster, Kuldip K Paliwal, and et al. 1997. Bidirectional recurrent neural networks. Signal Processing, IEEE Transactions on, 45(11):2673–2681. find
  11. Hristo Tanev, Vanni Zavarella, Jens Linge, Mijail Kabadjov, Jakub Piskorski, Martin Atkinson, and Ralf Steinberger. 2009. Exploiting machine learning techniques to build an event extraction system for portuguese and spanish. Linguamática, 1(2):55– 66. find
  12. Heng Ji, Ralph Grishman, and et al. 2008. Refining event extraction through cross-document inference. In ACL, pages 254–262. find our card
  13. Yann LeCun, Yoshua Bengio, and et al. 1995. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10). find
  14. Duyu Tang, Bing Qin, and Ting Liu. 2015a. Document modeling with gated recurrent neural network for sentiment classification. EMNLP. find our card
  15. Qi Li and Heng Ji. 2014. Incremental joint extraction of entity mentions and relations. In Proceedings of the Association for Computational Linguistics. find our card
  16. Duyu Tang, Bing Qin, and Ting Liu. 2015b. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1422–1432. find our card
  17. Qi Li, Heng Ji, and Liang Huang. 2013. Joint event extraction via structured prediction with global features. In ACL (1), pages 73–82. find our card
  18. Matthew D Zeiler. 2012. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. find our card
  19. Jiwei Li, Dan Jurafsky, and Eudard Hovy. 2015a. When are tree structures necessary for deep learning of representations? arXiv preprint arXiv:1503.00185. find
  20. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. 2014. Relation classification via convolutional deep neural network. In COLING, pages 2335–2344. find our card
  21. Jiwei Li, Minh-Thang Luong, and Dan Jurafsky. 2015b. A hierarchical neural autoencoder for paragraphs and documents. arXiv preprint arXiv:1506.01057. find
  22. Shasha Liao, Ralph Grishman, and et al. 2010. Using document level cross-event inference to improve event extraction. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 789–797. Association for Computational Linguistics. find our card
  23. Ting Liu, Wanxiang Che, and Zhenghua Li. 2011. Language technology platform. Journal of Chinese Information Processing, 25(6):53–62. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/feng-2016-language-independent-neural-network-event/