Elamur Research⌕
☰
home / papers / 2019 / Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs

Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs

Paper · 2019 · Fenia Christopoulou, Makoto Miwa and Sophia Ananiadou

Summary

The paper addresses document-level relation extraction (RE), where a relation between two entities often appears only across sentences, through chains of mentions. Earlier graph-based neural models use words as nodes and build entity-pair representations from the two target nodes alone. The authors argue that a relation is better captured by a representation specific to each pair. They propose EoG, an edge-oriented graph model. Each sentence is encoded with a BiLSTM. A partially connected document graph is then built whose nodes are mentions, entities and sentences rather than words, linked by simple heuristic rules. An iterative inference step combines existing edges into paths, producing entity-to-entity edge representations that are classified into relation types. Multi-instance learning over mentions is handled inside the graph. The model was tested on two biomedical document-level datasets, one for chemical-disease (CDR) and one for gene-disease associations. It reaches state-of-the-art results on the manually annotated CDR corpus without any syntactic dependency tools. The work matters because it shows that edge-centric graph reasoning can model intra- and inter-sentence relations together.

Key points

  • The architecture has four layers: sentence encoding (BiLSTM), graph construction, edge-based iterative inference, and classification.
  • The graph has three node types (mention, entity, sentence), each built by averaging its components and adding a node-type embedding. Entity nodes are never linked directly. Mention-mention edges exist only within the same sentence and include context and distance embeddings.
  • Analysis suggests that document-level information also improves intra-sentence relation detection, raising precision and F1.
  • Error analysis on the CDR development set found three main causes of missed inter-sentence pairs: entities joined by conjunctions or commas, missing coreference links, and incomplete entity linking, where synonyms received different KB IDs.
  • The source code is public at github.com/fenchri/edge-oriented-graph.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of ManchesterManchester · United Kingdom30 works · 43,942 citations🎓Toyota Technological InstituteNagoya · Japan6 works · 3,321 citations🔬National Institute of Advanced Industrial Science and TechnologyTsukuba · Japan13 works · 2,747 citations🔬Artificial Intelligence Research CenterTokyo · Japan8 works · 1,556 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Information extraction · 109 works

Cite

Cite as Fenia Christopoulou, Makoto Miwa and Sophia Ananiadou. 2019. Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1498

@inproceedings{christopoulou2019connecting,
  title = {{Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs}},
  author = {Fenia Christopoulou, Makoto Miwa and Sophia Ananiadou},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1498},
  url = {https://doi.org/10.18653/v1/d19-1498}
}

References

44 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michele Banko, Michael J Cafarella, Stephen Soderland, Matt Broadhead, and Oren Etzioni. 2007. Open information extraction from the web. In Proceedings of the International Joint Conference on Artifical Intelligence, pages 2670–2676. Morgan Kaufmann Publishers Inc.
  2. Karsten M Borgwardt and Hans-Peter Kriegel. 2005. Shortest-path kernels on graphs. In Proceedings of the IEEE International Conference on Data Mining, pages 74–81. IEEE Computer Society. find
  3. Billy Chiu, Gamal Crichton, Anna Korhonen, and Sampo Pyysalo. 2016. How to train good word embeddings for biomedical nlp. In Proceedings of the BioNLP workshop, pages 166–174. find
  4. Fenia Christopoulou, Makoto Miwa, and Sophia Ananiadou. 2018. A walk-based model on entity graphs for relation extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Volume 2), pages 81–88. Association for Computational Linguistics. find
  5. Jinghang Gu, Longhua Qian, and Guodong Zhou. 2016. Chemical-induced disease relation extraction with various linguistic features. Database. find
  6. Jinghang Gu, Fuqing Sun, Longhua Qian, and Guodong Zhou. 2017. Chemical-induced disease relation extraction via convolutional neural network. Database. find
  7. Pankaj Gupta, Subburam Rajaram, Hinrich Schtze, and Thomas Runkler. 2019. Neural relation extraction within and across sentence boundaries. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6513–6520. find
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  9. Robin Jia, Cliff Wong, and Hoifung Poon. 2019. Document-level n-ary relation extraction with multi-scale representation learning. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1), pages 3693– 3704.… find
  10. Xiaotian Jiang, Quan Wang, Peng Li, and Bin Wang. 2016. Relation extraction with multi-instance multi-label convolutional neural networks. In Proceedings of the International Conference on Computational Linguistics: Technical Papers, pages 1471– 1480. The COLING Organizing Committee. find
  11. Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations. find
  12. Haodi Li, Ming Yang, Qingcai Chen, Buzhou Tang, Xiaolong Wang, and Jun Yan. 2018. Chemical-induced disease extraction via recurrent piecewise convolutional neural networks. BMC medical informatics and decision making, 18(2):60. find
  13. Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sci-aky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Thomas C Wiegers, and Zhiyong Lu. 2016a. Biocreative v cdr task corpus: a resource for chemical disease relation extraction. Database. find
  14. Zhiheng Li, Zhihao Yang, Hongfei Lin, Jian Wang, Yingyi Gui, Yin Zhang, and Lei Wang. 2016b. CIDextractor: A chemical-induced disease relation extraction system for biomedical literature. In IEEE International Conference on Bioinformatics and Biomedicine, pages 994–1001. IEEE. find
  15. Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Volume 1), pages 2124– 2133. find
  16. Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019. A general framework for information extraction using dynamic span graphs. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo…
  17. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (Volume 2), pa… find
  18. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Volume 1), pages 1105–1116. Association for Computational Linguistics.
  19. Dat Quoc Nguyen and Karin Verspoor. 2018. Convolutional neural networks for chemical-disease relation extraction are improved with character-based word embeddings. In Proceedings of the BioNLP workshop, pages 129–136. Association for Computational Linguistics. find
  20. Thien Huu Nguyen and Ralph Grishman. 2015. Relation extraction: Perspective from convolutional neural networks. In Proceedings of the Workshop on Vector Space Modeling for Natural Language Processing, pages 39–48.
  21. Nagesh C Panyam, Karin Verspoor, Trevor Cohn, and Kotagiri Ramamohanarao. 2018. Exploiting graph kernels for high performance biomedical relation extraction. Journal of biomedical semantics, 9(1):7. find
  22. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in pytorch. In NIPS-W. find
  23. Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih. 2017. Cross-sentence n-ary relation extraction with graph lstms. Transactions of the Association for Computational Linguistics, 5:101–115.
  24. Yifan Peng, Chih-Hsuan Wei, and Zhiyong Lu. 2016. Improving chemical disease relation extraction with rich features and weakly labeled data. Journal of cheminformatics, 8(1):53. find
  25. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Empirical Methods in Natural Language Processing, pages 1532–1543. Association for Computational Linguistics.
  26. Janet Piñero, Àlex Bravo, Núria Queralt-Rosinach, Alba Gutiérrez-Sacristán, Jordi Deu-Pons, Emilio Centeno, Javier Garcı́a-Garcı́a, Ferran Sanz, and Laura I Furlong. 2016. Disgenet: a comprehensive platform integrating information on human disease-associated genes and variants. Nucleic acids re… find
  27. Chris Quirk and Hoifung Poon. 2017. Distant supervision for relation extraction beyond the sentence boundary. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (Volume 1), pages 1171–1182. Association for Computational Linguistics.
  28. Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 148–163. Springer. find
  29. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11):2673–2681. find
  30. Wei Shen, Jianyong Wang, and Jiawei Han. 2014. Entity linking with a knowledge base: Issues, techniques, and solutions. IEEE Transactions on Knowledge and Data Engineering, 27(2):443–460. find
  31. Gaurav Singh and Parminder Bhatia. 2019. Relation extraction using explicit context conditioning. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1442–1447. Associati… find
  32. Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. N-ary relation extraction using graph-state lstm. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 2226–2235. Association for Computational Linguistics. find
  33. Shikhar Vashishth, Rishabh Joshi, Sai Suman Prayaga, Chiranjib Bhattacharyya, and Partha Talukdar. 2018. Reside: Improving distantly-supervised neural relation extraction using side information. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1257–1266. As… find
  34. Patrick Verga, Emma Strubell, and Andrew McCallum. 2018. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volum…
  35. Linlin Wang, Zhu Cao, Gerard de Melo, and Zhiyuan Liu. 2016. Relation classification via multi-level attention CNNs. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Volume 1), pages 1298–1307. Association for Computational Linguistics.
  36. Ye Wu, Ruibang Luo, Henry CM Leung, Hing-Fung Ting, and Tak-Wah Lam. 2019. Renet: A deep learning approach for extracting gene-disease associations from literature. In International Conference on Research in Computational Molecular Biology, pages 272–284. Springer. find
  37. Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. 2015. Classifying relations via long short term memory networks along shortest dependency paths. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 1785–1794. Association for Computational Linguistics…
  38. Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1753–1762. find
  39. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of the International Conference on Computational Linguistics: Technical Papers, pages 2335–2344. Dublin City University and Association for Computationa…
  40. Wenyuan Zeng, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2017. Incorporating relation paths in neural relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1768–1777. Association for Computational Linguistics. find
  41. Yuhao Zhang, Peng Qi, and Christopher D Manning. 2018. Graph convolution over pruned dependency trees improves relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 2205–2215. Association for Computational Linguistics.
  42. Documents Positive pairs Intra Inter Negative pairs Entities Chemical Disease Mentions Chemical Disease Wei Zheng, Hongfei Lin, Zhiheng Li, Xiaoxia Liu, Zhengguang Li, Bo Xu, Yijia Zhang, Zhihao Yang, and Jian Wang. 2018. An effective neural model extracting document level chemical-induced disease r… find
  43. Documents Positive pairs Intra Inter Negative pairs Entities Gene Disease Mentions Gene Disease Peng Zhou, Jiaming Xu, Zhenyu Qi, Hongyun Bao, Zhineng Chen, and Bo Xu. 2018. Distant supervision for relation extraction with hierarchical selective attention. Neural Networks, 108:240–247. find
  44. A Datasets http://www.nactem.ac.uk/y-matsu/ geniass/ 7 http://www.nactem.ac.uk/GENIA/tagger/Test 500 1,066 747 319 4,138 1,467 1,965 1,507 1,864 1,434 1,988 5,162 4,252 5,307 4,328 5,370 4,430 Train 23,353 36,079 30,905 5,174 96,399 Dev 5,839 8,762 7,558 1,204 24,362 Test 1,000 1,502 1,305 197 3,720… ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/christopoulou-2019-connecting-dots-document-level-neural/