Elamur Research⌕
☰
home / papers / 2019 / Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network

Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network

Paper · 2019 · Sunil Kumar Sahu, Fenia Christopoulou, Makoto Miwa et al.

Summary

Relations between named entities in scientific text often cross sentence boundaries. Tools built for single sentences, such as one dependency tree or a sequential RNN/CNN encoder, have trouble linking evidence that sits far apart. The authors tackle document-level relation extraction by turning an entire abstract into one graph whose nodes are words. The graph uses several labelled edge kinds: syntactic dependencies inside sentences, coreference links, connections between the roots of neighbouring sentences, links between consecutive words, and self-loops. A labelled edge GCNN encodes this graph. Bi-affine pairwise scoring under a multi-instance learning scheme then pools evidence from every mention pair of the target entities. The model is tested on the CDR corpus and on CHR, a newly built, distantly supervised corpus of chemical reactant–product relations taken from PubMed abstracts. It outperforms the authors' CNN and RNN baselines on both corpora. On CDR it comes close to the strongest earlier systems while using a single unified model. Ablation shows that every edge type helps cross-sentence pairs, which supports the use of richer document graphs in biomedical text mining.

Key points

  • Graph construction: syntactic edges come from the Enju parser and coreference edges from Stanford CoreNLP. Text was preprocessed with the GENIA Sentence Splitter and GENIA tagger. Separate parameters are kept only for the top-N most frequent edge types, and N=4 worked best on the CDR dev set.
  • CHR resource: built using the Thalia search back-end and the Biochem4j graph database. It contains 7,298 / 1,182 / 3,614 train/dev/test articles and is released at nactem.ac.uk/CHR.
  • CDR results: GCNN reaches F1 58.6 (P 52.8, R 66.0). This is 1.6 points below Gu et al. (2017), which scores 60.2 with separate intra- and inter-sentence models. It is close to Li et al. (2018) at 59.1 and above Verga et al. (2018) at 55.5.
  • CHR results: GCNN reaches F1 87.5, compared with 86.4 for RNN-RE and 84.1 for CNN-RE.
  • Ablation (CDR dev): removing any edge type lowers inter-sentence F1, which is 36.90 for the full model. Dropping self-node edges causes the largest fall, to 33.20.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ManchesterManchester · United Kingdom30 works · 43,942 citations🎓Toyota Technological InstituteNagoya · Japan6 works · 3,321 citations🔬National Institute of Advanced Industrial Science and TechnologyTsukuba · Japan13 works · 2,747 citations🔬Artificial Intelligence Research CenterTokyo · Japan8 works · 1,556 citations

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputer Vision and Pattern Recognition · 1,282 worksBiochemistry, Genetics and Molecular Biology · 116 worksMolecular Biology · 114 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 109 works

Cite

Cite as Sunil Kumar Sahu, Fenia Christopoulou, Makoto Miwa and Sophia Ananiadou. 2019. Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1423

@inproceedings{sahu2019inter,
  title = {{Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network}},
  author = {Sunil Kumar Sahu, Fenia Christopoulou, Makoto Miwa and Sophia Ananiadou},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1423},
  url = {https://doi.org/10.18653/v1/p19-1423}
}

References

35 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aron Culotta and Jeffrey Sorensen. 2004. Dependency tree kernels for relation extraction. In Proceedings of Annual Meeting on Association for Computational Linguistics, pages 423–430. Association for Computational Linguistics.
  2. Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. 2016. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pages 3844–3852.
  3. Jinghang Gu, Fuqing Sun, Longhua Qian, and Guodong Zhou. 2017. Chemical-induced disease relation extraction via convolutional neural network. Database, 2017:1–12. find
  4. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 1746–1751. Association for Computational Linguistics.
  5. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations. find
  6. Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations.
  7. Haodi Li, Ming Yang, Qingcai Chen, Buzhou Tang, Xiaolong Wang, and Jun Yan. 2018. Chemical-induced disease extraction via recurrent piecewise convolutional neural networks. BMC Medical Informatics and Decision Making, 18(2):60. find
  8. Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of Annual Meeting of the Association for Computational Linguistics, volume 1, pages 2124–2133. Association for Computational Linguistics. find
  9. Yang Liu, Furu Wei, Sujian Li, Heng Ji, Ming Zhou, and Houfeng Wang. 2015. A dependency-based neural network for relation classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processi… find
  10. Xuezhe Ma, Zhengzhong Liu, and Eduard Hovy. 2016. Unsupervised ranking model for entity coreference resolution. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1012–1018. Association for Computational…
  11. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60. Association…
  12. Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 1506–1515. Association for Computational Linguistics.
  13. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using lstms on sequences and tree structures. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1105–1116. Association for Computational Linguistics.
  14. Yusuke Miyao and Jun’ichi Tsujii. 2008. Feature forest models for probabilistic HPSG parsing. Computational Linguistics, 34(1):35–80.
  15. Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih. 2017. Cross-sentence n-ary relation extraction with Graph LSTMs. Transactions of the Association for Computational Linguistics, 5:101–115.
  16. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1532–1543. Association for Computational Linguistics.
  17. Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 148–163. Springer. find
  18. Sunil Kumar Sahu and Ashish Anand. 2018. Drug-drug interaction extraction from biomedical texts using long short-term memory network. Journal of Biomedical Informatics, 86:15 – 24. find
  19. Cıcero Nogueira dos Santos, Bing Xiang, and Bowen Zhou. 2015. Classifying relations by ranking with convolutional neural networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pages 626–63…
  20. Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. N-ary relation extraction using graph-state LSTM. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 2226–2235. Association for Computational Linguistics. on freebase via relation extraction and textu… find
  21. Axel J Soto, Piotr Przybyła, and Sophia Ananiadou. 2018. Thalia: Semantic search engine for biomedical abstracts. Bioinformatics. find
  22. Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 455–465. Ass… find
  23. Neil Swainston, Riza Batista-Navarro, Pablo Carbonell, Paul D Dobson, Mark Dunstan, Adrian J Jervis, Maria Vinaixa, Alan R Williams, Sophia Ananiadou, Jean-Loup Faulon, et al. 2017. biochem4j: Integrated and extensible biochemical knowledge through graph databases. PloS one, 12(7):e0179130. find
  24. Michihiro Yasunaga, Rui Zhang, Kshitijh Meelu, Ayush Pareek, Krishnan Srinivasan, and Dragomir Radev. 2017. Graph-based neural multi-document summarization. In Proceedings of the 21st Conference on Computational Natural Language Learning, pages 452–462. Association for Computational Linguistics.
  25. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation Classification via Convolutional Deep Neural Network. In Proceedings of the International Conference on Computational Linguistics, pages 2335–2344.
  26. MUNEEB TH, Sunil Sahu, and Ashish Anand. 2015. Evaluating distributed word representations for capturing semantics of biomedical concepts. In Proceedings of BioNLP, pages 158–163. Association for Computational Linguistics. find
  27. Yuhao Zhang, Peng Qi, and Christopher D Manning. 2018. Graph convolution over pruned dependency trees improves relation extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2205–2215. Association for Computational Linguistics.
  28. Yoshimasa Tsuruoka, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun’ichi Tsujii. 2005. Developing a robust part-of-speech tagger for biomedical text. In Panhellenic Conference on Informatics, pages 382–392. Springer. find
  29. Huiwei Zhou, Huijie Deng, Long Chen, Yunlong Yang, Chen Jia, and Degen Huang. 2016a. Exploiting syntactic and semantics information for chemical–disease relation extraction. Database, 2016:1– 12. find
  30. Shikhar Vashishth, Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. 2018. Dating documents using graph convolution networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1605–1615. Association for Computational Linguistics. find
  31. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016b. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 207–212. Association for Comput…
  32. Patrick Verga, Emma Strubell, and Andrew McCallum. 2018. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 872…
  33. Chih-Hsuan Wei, Yifan Peng, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Jiao Li, Thomas C Wiegers, and Zhiyong Lu. 2015. Overview of the BioCreative V chemical disease relation (CDR) task. In Proceedings of the fifth BioCreative challenge evaluation workshop, pages 154–166. find
  34. Jun Xu, Yonghui Wu, Yaoyun Zhang, Jingqi Wang, Hee-Jin Lee, and Hua Xu. 2016a. CD-REST: a system for extracting chemical-induced disease relation in literature. Database, 2016:1–10. find
  35. Kun Xu, Siva Reddy, Yansong Feng, Songfang Huang, and Dongyan Zhao. 2016b. Question answering find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sahu-2019-inter-sentence-relation-extraction-document/