Elamur Research⌕
☰
home / papers / 2014 / Relation Classification via Convolutional Deep Neural Network

Relation Classification via Convolutional Deep Neural Network

Summary

The paper addresses relation classification: deciding which semantic relation holds between two marked nominals in a sentence. Earlier supervised systems, whether feature-based or kernel-based, depend on outputs of NLP tools such as POS taggers and parsers, so those tools' errors carry over into the classifier. The authors instead use a convolutional deep neural network that reads raw tokens with no complicated preprocessing.

Each word is mapped to a pretrained embedding from Turian et al. (2010). Two kinds of features are then combined:

  • Lexical features: embeddings of the marked nouns, their neighbouring tokens, and WordNet hypernyms.
  • Sentence-level features: a convolution over token windows, followed by max-over-time pooling. Each token's representation is enriched with position features that encode its relative distance to both target nouns.

The concatenated vector feeds a softmax classifier.

On the SemEval-2010 Task 8 dataset, the model reaches a macro-averaged F1 of 82.7. This beats SVM/MaxEnt systems built on rich hand-crafted resources and the parser-dependent RNN and MVRNN models. The result suggests that learned features can replace elaborate tool-based pipelines.

Key points

  • The model was evaluated on SemEval-2010 Task 8 using the corpus's Perl-based scorer and macro-averaged F1.
  • Main comparison:
    • The proposed model scores 82.7 F1.
    • MVRNN scores 82.4 with POS, NER and WordNet added.
    • The best traditional SVM system scores 82.2; it relies on many resources, including dependency parses, FrameNet, Google n-grams and TextRunner.
    • RNN scores 77.6 with POS, NER and WordNet added.
    • A t-test (p ≤ 0.05) indicates the improvement is significant.
  • Hyperparameters: window size 3, word dimension 50, distance dimension 5, hidden layers of 200 and 100 units, learning rate 0.01. Windows wider than 3 brought no gain.
  • Lexical ablation: F1 rises step by step as feature sets are added: 34.7 (nouns only), 53.1, 59.4, 65.9, and 73.3 (with WordNet hypernyms).
  • Sentence-level ablation: word features alone give 69.7; adding position features raises this to 78.9, roughly 9.2 points. Combining everything gives 82.7.

Interesting ideas

  • I1. Position features: Each token gets a learned embedding of its distance to each target entity. This tells a sentence-level encoder which pair of words the decision concerns. The trick transfers to any pairwise task over one sentence, such as entity linking, argument detection or event-role labelling.
  • I2. Two-level feature design: The model joins a "local" vector (the entities and their immediate neighbours) with a "global" vector (a pooled convolution over the whole sentence). This is a cheap way to give a classifier both focused and contextual evidence.
  • I3. Avoiding pipeline errors: The model deliberately skips parsers and taggers in favour of pretrained embeddings. This is worth considering for languages or domains where parsers are weak, since errors in upstream tools then cannot propagate into the classifier.
  • I4. Max-over-time pooling: Pooling turns sentences of any length into a fixed-size vector. It is a simple general-purpose way to feed variable-length text into a standard classifier.
  • I5. Incremental ablation: Adding feature groups one at a time and reporting F1 at each step shows clearly which inputs matter. This makes it a useful evaluation template for justifying the parts of a model.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

327 papers of our library cite it (of the 140884 whose bibliographies we read); 20 of them have a page here, 307 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zeng2014relation,
  title = {{Relation Classification via Convolutional Deep Neural Network}},
  author = {Zeng, Daojian and Liu, Kang and Lai, Siwei and Zhou, Guangyou and Zhao, Jun},
  booktitle = {Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers},
  year = {2014},
  url = {https://aclanthology.org/C14-1220/}
}

References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nguyen Bach and Sameer Badaskar. 2007. A review of relation extraction. Literature review for Language and Statistics II. find
  2. Razvan C. Bunescu and Raymond J. Mooney. 2005. A shortest path dependency kernel for relation extraction. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 724–731. find our card
  3. Jinxiu Chen, Donghong Ji, Chew Lim Tan, and Zhengyu Niu. 2005. Unsupervised feature selection for relation extraction. In Proceedings of the International Joint Conference on Natural Language Processing, pages 262– 267. find
  4. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493–2537. find our card
  5. Zellig Harris. 1954. Distributional structure. Word, 10(23):146–162. find
  6. Takaaki Hasegawa, Satoshi Sekine, and Ralph Grishman. 2004. Discovering relations among named entities from large corpora. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, pages 415–422. find our card
  7. Kazuma Hashimoto, Makoto Miwa, Yoshimasa Tsuruoka, and Takashi Chikayama. 2013. Simple customization of recursive neural networks for semantic relation classification. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1372–1376. find
  8. Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó. Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2010. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In Proceedings of the 5th Internationa… find our card
  9. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volum… find
  10. Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics on Interactive poster and demonstration sessions. find
  11. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2… find
  12. Raymond J Mooney and Razvan C Bunescu. 2005. Subsequence kernels for relation extraction. In Advances in neural information processing systems, pages 171–178. find
  13. Longhua Qian, Guodong Zhou, Fang Kong, Qiaoming Zhu, and Peide Qian. 2008. Exploiting constituent dependencies for tree kernel-based semantic relation extraction. In Proceedings of the 22nd International Conference on Computational Linguistics, pages 697–704. find
  14. Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III, pages 148–163. find
  15. Richard Socher, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201–1211. find our card
  16. Fabian M. Suchanek, Georgiana Ifrim, and Gerhard Weikum. 2006. Combining linguistic and statistical analysis to extract relations from web documents. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 712–717. find
  17. Shingo Takamatsu, Issei Sato, and Hiroshi Nakagawa. 2012. Reducing wrong labels in distant supervision for relation extraction. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1, pages 721–729. find
  18. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 384–394. find our card
  19. Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2003. Kernel methods for relation extraction. The Journal of Machine Learning Research, 3:1083–1106. find our card
  20. GuoDong Zhou, Su Jian, Zhang Jie, and Zhang Min. 2005. Exploring various knowledge in relation extraction. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 427–434. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zeng-2014-relation-classification-convolutional-deep-neural/