Elamur Research⌕
☰
home / papers / 2017 / Cross-Sentence N-ary Relation Extraction with Graph LSTMs

Cross-Sentence N-ary Relation Extraction with Graph LSTMs

Summary

Most relation extraction research handles only two entities appearing in a single sentence. In biomedicine, however, facts often involve several entities spread across sentences. One example is a drug that works against tumors carrying a specific mutation in a specific gene. Peng et al. propose a framework built on graph LSTMs to capture such relations. Each document becomes a graph whose edges encode word adjacency, syntactic dependencies and discourse links, including links between neighbouring sentences. The LSTM learns a contextual vector for every word, and the vectors of the target entities are concatenated and passed to a relation classifier. This design handles any number of arguments and allows joint training with simpler sub-relations. The authors tested the approach in two precision-medicine settings, using both supervised learning and distant supervision. It beat other neural variants and a carefully engineered feature-based baseline. Extracting across sentence boundaries over PubMed-scale text produced a much larger knowledge base, which matters for curating cancer-treatment knowledge.

Key points

  • The core task is a ternary drug–gene–mutation relation: deciding whether tumors with mutation v in gene g respond to drug d.
  • The document graph may contain cycles. To make training tractable, the authors split it into two directed acyclic graphs, one for a forward pass and one for a backward pass. The paper also compares two strategies for parametrizing the recurrent units over many edge types.
  • The graph formulation includes chain and tree LSTMs as special cases. Adding syntax helped noticeably, especially when parse quality was high.
  • Multi-task learning with binary sub-relations, such as drug–mutation with the gene left unspecified, gave further accuracy gains.
  • When trained with distant supervision derived from a limited collection of known interactions, cross-sentence extraction roughly tripled the yield compared with single-sentence extraction. Manual checks indicated high precision.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

110 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 101 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00049

@article{peng2017cross,
  title = {{Cross-Sentence N-ary Relation Extraction with Graph LSTMs}},
  author = {Peng, Nanyun and Poon, Hoifung and Quirk, Chris and Toutanova, Kristina and Yih, Wen-tau},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2017},
  doi = {10.1162/tacl\_a\_00049},
  url = {https://doi.org/10.1162/tacl_a_00049}
}

References

59 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Collin Baker, Charles Fillmore, and John Lowe. 1998. The Berkeley FrameNet project. In Proceedings of the Thirty-Sixth Annual Meeting of the Association for Computational Linguistics and Seventeenth International Conference on Computational Linguistics. find our card
  2. Dane Bell, Gustave Hahn-Powell, Marco A. Valenzuela-Escarcega, and Mihai Surdeanu. 2016. An investigation of coreference phenomena in the biomedical domain. In Proceedings of the Tenth Edition of the Language Resources and Evaluation Conference. find
  3. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2). find
  4. Elizabeth Boschee, Ralph Weischedel, and Alex Zamanian. 2005. Automatic information extraction. In Proceedings of the International Conference on Intelligence Analysis. find
  5. Razvan C Bunescu and Raymond J Mooney. 2005. A shortest path dependency kernel for relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
  6. Rui Cai, Xiaodong Zhang, and Houfeng Wang. 2016. Bidirectional recurrent convolutional neural network for relation classification. In Proceedings of the Fifty-Fourth Annual Meeting of the Association for Computational Linguistics. find
  7. Rich Caruana, Steve Lawrence, and Lee Giles. 2001. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In Proceedings of The Fifteenth Annual Conference on Neural Information Processing Systems. find
  8. Rich Caruana. 1998. Multitask learning. In Learning to learn. find
  9. Springer. Yee Seng Chan and Dan Roth. 2010. Exploiting background knowledge for relation extraction. In Proceedings of the Twenty-Third International Conference on Computational Linguistics. find
  10. Nancy Chinchor. 1998. Overview of MUC-7/MET-2. Technical report, Science Applications International Corporation, San Diego, CA. find
  11. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the Twenty-Fifth International Conference on Machine learning. find
  12. Mark Craven and Johan Kumlien. 1999. Constructing biological knowledge bases by extracting information from text sources. In Proceedings of the Seventh International Conference on Intelligent Systems for Molecular Biology. find
  13. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the Fifth International Conference on Language Resources and Evaluation. find our card
  14. Rodrigo Dienstmann, In Sock Jang, Brian Bot, Stephen Friend, and Justin Guinney. 2015. Database of genomic biomarkers for cancer drugs and clinical tar-getability in solid tumors. Cancer Discovery, 5. find
  15. Nicholas FitzGerald, Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. 2015. Semantic role labeling with neural network factors. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  16. Matthew Gerber and Joyce Y. Chai. 2010. Beyond Nom-Bank: A study of implicit arguments for nominal predicates. In Proceedings of the Forty-Eighth Annual Meeting of the Association for Computational Linguistics. find
  17. Alan Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In Proceedings of The Thirty-Eighth IEEE International Conference on Acoustics, Speech and Signal Processing. find
  18. Zhou GuoDong, Su Jian, Zhang Jie, and Zhang Min. 2005. Exploring various knowledge in relation extraction. In Proceedings of the Forty-Third Annual Meeting of the Association for Computational Linguistics. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8… find our card
  19. Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of the Forty-Second Annual Meeting of the Association for Computational Linguistics, Demonstration Sessions. find
  20. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009. Overview of BioNLP’09 shared task on event extraction. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing: Shared Task. find our card
  21. Heeyoung Lee, Angel Chang, Yves Peirsman, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2013. Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics, 39(4). find our card
  22. Hong Li, Sebastian Krause, Feiyu Xu, Andrea Moro, Hans Uszkoreit, and Roberto Navigli. 2015. Improvement of n-ary relation extraction by adding lexical semantics to distant-supervision rule learning. In Proceedings of the Seventh International Conference on Agents and Artificial Intelligence. find
  23. Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2016. Gated graph sequence neural networks. In Proceedings of the Fourth International Conference on Learning Representations. find
  24. Xiaodan Liang, Xiaohui Shen, Jiashi Feng, Liang Lin, and Shuicheng Yan. 2016. Semantic object parsing with graph LSTM. In Proceedings of European Conference on Computer Vision. find
  25. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of the Fifty-Second Annual Meeting of the Association for Computational Linguistics: System Demonstrations. find our card
  26. Ryan McDonald, Fernando Pereira, Seth Kulick, Scott Winters, Yang Jin, and Pete White. 2005. Simple algorithms for complex relation extraction with applications to biomedical IE. In Proceedings of the Forty-Third Annual Meeting on Association for Computational Linguistics. find
  27. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the Forty-Seventh Annual Meeting of the Association for Computational Linguistics and the Fourth International Joint Conference on… find
  28. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of the Fifty-Fourth Annual Meeting of the Association for Computational Linguistics. find our card
  29. Raymond J Mooney and Razvan C Bunescu. 2005. Subsequence kernels for relation extraction. In Proceedings of The Nineteen Annual Conference on Neural Information Processing Systems. find
  30. Thien Huu Nguyen and Ralph Grishman. 2014. Employing word representations and regularization for domain adaptation of relation extraction. In Proceedings of the Fifty-Second Annual Meeting of the Association for Computational Linguistics. find
  31. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1). find our card
  32. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In Proceedings of The Thirtieth International Conference on Machine Learning. find
  33. Nanyun Peng and Mark Dredze. 2016. Improving named entity recognition for chinese social media with word segmentation representation learning. In Proceedings of the Fifty-Fourth Annual Meeting of the Association for Computational Linguistics. find
  34. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  35. Hoifung Poon, Chris Quirk, Charlie DeZiel, and David Heckerman. 2014. Literome: PubMed-scale genomic knowledge base in the cloud. Bioinformatics, 30(19). find
  36. Hoifung Poon, Kristina Toutanova, and Chris Quirk. 2015. Distant supervision for cancer pathway extraction from text. In Pacific Symposium on Biocomputing. find
  37. Longhua Qian, Guodong Zhou, Fang Kong, Qiaoming Zhu, and Peide Qian. 2008. Exploiting constituent dependencies for tree kernel-based semantic relation extraction. In Proceedings of the Twenty-Second International Conference on Computational Linguistics. find
  38. Chris Quirk and Hoifung Poon. 2017. Distant supervision for relation extraction beyond the sentence boundary. In Proceedings of the Fifteenth Conference on European chapter of the Association for Computational Linguistics. find
  39. Chris Quirk, Pallavi Choudhury, Jianfeng Gao, Hisami Suzuki, Kristina Toutanova, Michael Gamon, Wen-tau Yih, and Lucy Vanderwende. 2012. MSR SPLAT, a language analysis toolkit. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… find
  40. Kevin Reschke, Martin Jankowiak, Mihai Surdeanu, Christopher D Manning, and Daniel Jurafsky. 2014. Event extraction using distant supervision. In Proceedings of Eighth edition of the Language Resources and Evaluation Conference. find
  41. Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In Proceedings of the Fifty-Fourth Annual Meeting of the Association for Computational Linguistics. find our card
  42. Cicero Nogueira dos Santos, Bing Xiang, and Bowen Zhou. 2015. Classifying relations by ranking with convolutional neural networks. In Proceedings of the Fifty-Third Annual Meeting of the Association for Computational Linguistics. find our card
  43. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2009. The graph neural network model. IEEE Transactions on Neural Networks, 20(1). find
  44. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. find our card
  45. Fabian M Suchanek, Georgiana Ifrim, and Gerhard Weikum. 2006. Combining linguistic and statistical analysis to extract relations from web documents. In Proceedings of the Twelfth International Conference on Knowledge Discovery and Data Mining. find
  46. Mihai Surdeanu and Ji Heng. 2014. Overview of the english slot filling track at the TAC2014 knowledge base population evaluation. In Proceedings of the U.S. National Institute of Standards and Technology Knowledge Base Population 2014 Workshop. find
  47. Kumutha Swampillai and Mark Stevenson. 2011. Extracting relations within and across sentences. In Proceedings of the Conference on Recent Advances in Natural Language Processing. find
  48. Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the Fifty-Third Annual Meeting of the Association for Computational Linguistics. find
  49. Theano Development Team. 2016. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints, abs/1605.02688. find our card
  50. Linlin Wang, Zhu Cao, Gerard de Melo, and Zhiyuan Liu. 2016. Relation classification via multi-level attention CNNs. In Proceedings of the Fifty-Fourth Annual Meeting of the Association for Computational Linguistics. find our card
  51. Michael Wick, Aron Culotta, and Andrew McCallum. 2006. Learning field compatibilities to extract database records from unstructured text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  52. Kun Xu, Yansong Feng, Songfang Huang, and Dongyan Zhao. 2015a. Semantic relation classification via convolutional neural networks with simple negative sampling. In Proceedings of Conference on Empirical Methods in Natural Language Processing. find
  53. Yan Xu, Lili Mou, Ge Li, Yunchuan Chen, Hao Peng, and Zhi Jin. 2015b. Classifying relations via long short term memory networks along shortest dependency paths. In Proceedings of Conference on Empirical Methods in Natural Language Processing. find our card
  54. Yan Xu, Ran Jia, Lili Mou, Ge Li, Yunchuan Chen, Yangyang Lu, and Zhi Jin. 2016. Improved relation classification by deep recurrent neural networks with data augmentation. In Proceedings of the Twenty-Sixth International Conference on Computational Linguistics. find
  55. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol Rutherford. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of the Conference on Computational Natural Language Learning, Shared Task. find our card
  56. Katsumasa Yoshikawa, Sebastian Riedel, Tsutomu Hirao, Masayuki Asahara, and Yuji Matsumoto. 2011. Coreference based event-argument relation extraction on biomedical text. Journal of Biomedical Semantics, 2(5). find
  57. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, Jun Zhao, et al. 2014. Relation classification via convolutional deep neural network. In Proceedings of the Twenty-Sixth International Conference on Computational Linguistics. find our card
  58. Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  59. Shu Zhang, Dequan Zheng, Xinchen Hu, and Ming Yang. 2015. Bidirectional long short-term memory networks for relation classification. In Proceedings of Twenty-Ninth Pacific Asia Conference on Language, Information and Computation. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/peng-2017-cross-sentence-n-ary-relation/