Elamur Research⌕
☰
home / papers / 2017 / Distant Supervision for Relation Extraction beyond the Sentence Boundary

Distant Supervision for Relation Extraction beyond the Sentence Boundary

Paper · 2017 · Chris Quirk and Hoifung Poon

Summary

Distant supervision has typically trained relation extractors only on entity pairs found in the same sentence. Many facts, especially in specialised scientific writing, are spread across neighbouring sentences. The authors present DISCREX, which they describe as the first distant-supervision method for cross-sentence relation extraction.

Each document becomes one graph that combines:

  • intra-sentence dependency arcs,
  • links between roots of consecutive sentences,
  • word-adjacency edges,
  • optional coreference and discourse edges.

Features come from several paths between the two entities, not just the shortest one. Candidate pairs are limited to minimal spans.

The task is drug-gene interaction extraction for precision medicine. Training uses only about 162 usable facts from the curated GDKD knowledge base, plus unlabeled PubMed Central text. Across roughly a million full-text articles, cross-sentence extraction about doubled the number of unique interactions found by single-sentence extraction, at comparable precision. This shows that relations crossing sentence boundaries are common and can be learned cheaply.

Key points

  • Supervision source: GDKD has 341 relations, but only 162 name specific drugs usable for training. Negatives are randomly sampled co-occurring pairs, kept roughly equal in number to the positives.
  • Best configuration: features from the 30 shortest paths per entity pair, with minimal-span candidates covering up to three sentences. Dropping the minimal-span filter lowered test accuracy from 81.7% to 79.1%.
  • Yield at p ≥ 0.5: cross-sentence extraction found 64,828 unique drug-gene pairs, versus 32,028 for single-sentence extraction. Absolute recall rose by 89–102% depending on the threshold.
  • Manual evaluation: 450 instances were judged, with annotators agreeing on 133 of 150 doubly-annotated cases.
    • Cross-sentence precision was 57% at p ≥ 0.5 and 61% at p ≥ 0.9, against a 23% random baseline.
    • Entity-linking mistakes accounted for a large share of errors.
  • Discourse tools gave little benefit: Stanford coreference and the CoNLL-2015 winning discourse parser (Wang and Lan) brought little gain. The coreference system often failed on biomedical text.

Interesting ideas

  • I1. One document graph for everything: representing a whole document as a single graph turns cross-sentence extraction into ordinary path-feature extraction. New signals such as coreference or rhetorical links can be added just as extra edge types, a design that could carry over to other low-resource or domain-specific corpora.
  • I2. Multiple paths instead of one: using the top-K paths, with dependency and word-adjacency edges allowed to interleave, guards against parser mistakes and wording variation. This is relevant wherever parsers are imperfect, such as smaller languages or noisy text.
  • I3. Edge-type weighting: penalising adjacency edges relative to dependency edges keeps informative syntactic sub-paths inside the top K. A harsh penalty matters most when few paths are kept, a cheap trick for any graph built from heterogeneous edges.
  • I4. Minimal-span candidates: keeping only the tightest span that contains an entity pair is a simple filter that removes many implausible long-distance candidates and measurably improves accuracy.
  • I5. Separating error sources in manual evaluation: the authors counted entity-linking errors apart from relation errors. This shows where a pipeline really fails, and here it pointed to entity linking as the main bottleneck, a useful evaluation practice for any knowledge-base population project.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States240 works · 123,537 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Biochemistry, Genetics and Molecular Biology · 116 works Molecular Biology · 114 works

🧭 Topics 2 Information extraction · 109 works Tokenization & segmentation · 77 works

Cite

Cite as Chris Quirk and Hoifung Poon. 2017. Distant Supervision for Relation Extraction beyond the Sentence Boundary. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{quirk2017distant,
  title = {{Distant Supervision for Relation Extraction beyond the Sentence Boundary}},
  author = {Chris Quirk and Hoifung Poon},
  booktitle = {Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers},
  year = {2017},
  doi = {10.18653/v1/e17-1110},
  url = {https://doi.org/10.18653/v1/e17-1110}
}

References

53 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michele Banko, Michael J. Cafarella, Stephen Soderland, Matt Broadhead, and Oren Etzioni. 2007. Open information extraction from the web. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, pages 2670–2676, Hyderabad, India. AAAI Press. our card
  2. Dane Bell, Gustave Hahn-Powell, Marco A. Valenzuela-Escarcega, and Mihai Surdeanu. 2016. An investigation of coreference phenomena in the biomedical domain. In Proceedings of LREC, pages 177–183. find
  3. Jari Björne, Juho Heimonen, Filip Ginter, Antti Airola, Tapio Pahikkala, and Tapio Salakoski. 2009. Extracting complex biological events with rich graph-based feature sets. In Proceedings of the BioNLP 2009 Workshop Companion Volume for Shared Task, pages 10–18, Boulder, Colorado, June. Association… find
  4. Nathanael Chambers and Dan Jurafsky. 2009. Unsupervised learning of narrative schemas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 602–610, Suntec, Si… find
  5. Jackie Chi Kit Cheung, Hoifung Poon, and Lucy Vanderwende. 2013. Probabilistic frame induction. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 837–846, Atlanta, Georgia, June. Association for Co… find
  6. Kevin Clark and Christopher D. Manning. 2015. Entity-centric coreference resolution with model stacking. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages… our card
  7. M. W. Craven, D. DiPasquo, D. Freitag, A. McCallum, T. Mitchell, K. Nigam, and S. Slattery. 1998. Learning to extract symbolic knowledge from the World Wide Web. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 509–516, Madison, WI. AAAI Press. find
  8. Mark Craven, Dan DiPasquo, Dayne Freitag, Andrew McCallum, Tom Mitchell, Kamal Nigam, and Sean Slattery. 2000. Learning to construct knowledge bases from the world wide web. Artificial Intelligence, 118:69–113. find
  9. Dipanjan Das and Noah A. Smith. 2011. Semi-supervised frame-semantic parsing for unknown predicates. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1435– 1444, Portland, Oregon, USA, June. Association for Computational L… our card
  10. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages… find
  11. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13–24, Baltimore, Maryland, June. Association for Computational Linguistics. our card
  12. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the Fifth International Conference on Language Resources and Evaluation, pages 449– 454, Genoa, Italy. ELRA. our card
  13. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009. Overview of BioNLP’09 Shared Task on event extraction. In Proceedings of the BioNLP 2009 Workshop Companion Volume for Shared Task, pages 1–9, Boulder, Colorado, June. Association for Computational Linguistics. our card
  14. Rodrigo Dienstmann, In Sock Jang, Brian Bot, Stephen Friend, and Justin Guinney. 2015. Database of genomic biomarkers for cancer drugs and clinical targetability in solid tumors. Cancer Discovery, 5:118–123. find
  15. Mitchell Koch, John Gilmer, Stephen Soderland, and S. Daniel Weld. 2014. Type-aware distantly supervised relation extraction with linked arguments. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1891–1901. Association for Computational Lingui… find
  16. Greg Durrett and Dan Klein. 2014. A joint model for entity analysis: Coreference, typing, and linking. Transactions of the Association for Computational Linguistics, 2:477–490. our card
  17. Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 511–521, Baltimore, Maryland, June. Association for Computati… our card
  18. Matt Gardner and Tom Mitchell. 2015. Efficient and expressive knowledge base completion using subgraph feature extraction. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1488–1498, Lisbon, Portugal, September. Association for Computational Linguistic… find
  19. Ni Lao, Tom Mitchell, and William W. Cohen. 2011. Random walk inference and learning in a large scale knowledge base. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 529–539, Edinburgh, Scotland, UK., July. Association for Computational Linguistics. find
  20. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s multi-pass sieve coreference resolution system at the conll-2011 shared task. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 28–… our card
  21. Aria Haghighi and Dan Klein. 2007. Unsupervised coreference resolution in a nonparametric bayesian model. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 848–855, Prague, Czech Republic, June. Association for Computational Linguistics. our card
  22. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP Natural Language Processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60, Baltimore, Ma… our card
  23. Hannaneh Hajishirzi, Leila Zilles, Daniel S. Weld, and Luke Zettlemoyer. 2013. Joint coreference resolution and named-entity linking with multi-pass Daniel Marcu. 2000. The Theory and Practice of Discourse Parsing and Summarization. The MIT Press, Cambridge, Massachusetts, November. find
  24. Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages… find
  25. Claire Nédellec, Robert Bossy, Jin-Dong Kim, Jung-Jae Kim, Tomoko Ohta, Sampo Pyysalo, and Pierre Zweigenbaum. 2013. Overview of BioNLP Shared Task 2013. In Proceedings of the BioNLP Shared Task 2013 Workshop, pages 1–7, Sofia, Bulgaria, August. Association for Computational Linguistics. find
  26. J. Nocedal and S. Wright. 2006. Numerical Optimization. Springer, New York, NY. find
  27. Ankur P. Parikh, Hoifung Poon, and Kristina Toutanova. 2015. Grounded semantic parsing for complex knowledge extraction. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 756– 766, Denver, Colorado… find
  28. Hoifung Poon and Pedro Domingos. 2008. Joint unsupervised coreference resolution with Markov logic. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 649–658, Honolulu, HI. ACL. our card
  29. Hoifung Poon and Pedro Domingos. 2009. Unsupervised semantic parsing. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1–10, Singapore. ACL. our card
  30. Hoifung Poon and Lucy Vanderwende. 2010. Joint inference for knowledge extraction from biomedical literature. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 813–821, Los Angeles, California, June. Assoc… find
  31. Canada, June. Association for Computational Linguistics. find
  32. Dragomir R. Radev and Rada Mihalcea. 2008. Networks and natural language processing. AI Magazine, 29(3):16–28. find
  33. Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nate Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher Manning. 2010. A multi-pass sieve for coreference resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 492–501, Cambridge, MA,… our card
  34. Altaf Rahman and Vincent Ng. 2009. Supervised models for coreference resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 968–977, Singapore, August. Association for Computational Linguistics. our card
  35. Marta Recasens, Marie-Catherine de Marneffe, and Christopher Potts. 2013. The life and death of discourse entities: Identifying singleton mentions. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages… our card
  36. Sebastian Riedel and Andrew McCallum. 2011. Fast and robust joint models for biomedical event extraction. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1–12, Edinburgh, Scotland, UK., July. Association for Computational Linguistics. find
  37. Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. 2013. Relation extraction with matrix factorization and universal schemas. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 74… find
  38. Hoifung Poon, Kristina Toutanova, and Chris Quirk. 2015. Distant supervision for cancer pathway extraction from text. In Pacific Symposium of Biocomputing, pages 121–131, Big Island of Hawaii. find
  39. Rebecca Sharp, Peter Jansen, Mihai Surdeanu, and Peter Clark. 2015. Spinning straw into gold: Using free text to train monolingual alignment models for non-factoid question answering. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics… find
  40. Chris Quirk, Pallavi Choudhury, Jianfeng Gao, Hisami Suzuki, Kristina Toutanova, Michael Gamon, Wen-tau Yih, Colin Cherry, and Lucy Vanderwende. 2012. MSR SPLAT, a language analysis toolkit. In Proceedings of the Demonstration Session at the Conference of the North American Chapter of the Associatio… find
  41. Hoifung Poon, Chris Quirk, Charlie DeZiel, and David Heckerman. 2014. Literome: Pubmed-scale genomic knowledge base in the cloud. Bioinformatics, 30(19):2840–2842. find
  42. Amarnag Subramanya, Slav Petrov, and Fernando Pereira. 2010. Efficient graph-based semi-supervised learning of structured tagging models. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 167–176, Cambridge, MA, October. Association for Computational Li… find
  43. Mihai Surdeanu and Heng Ji. 2014. Overview of the English Slot Filling Track at the TAC2014 Knowledge Base Population evaluation. In Proceedings of the TAC-KBP 2014 Workshop, pages 1–15, Gaithersburg, Maryland, USA. find
  44. Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 455– 4… find
  45. Fei Wu and Daniel S. Weld. 2007. Autonomously semantifying wikipedia. In Proceedings of the Sixteenth ACM Conference on Conference on Information and Knowledge Management, CIKM ’07, pages 41–50, New York, NY, USA. ACM. find
  46. Hao Zheng, Zhoujun Li, Senzhang Wang, Zhao Yan, and Jianshe Zhou. 2016. Aggregating inter-sentence information to enhance relation extraction. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, pages 3108–3114. AAAI Press. find
  47. Mihai Surdeanu, Tom Hicks, and Marco Antonio Valenzuela-Escarcega. 2015. Two practical rhetorical structure theory parsers. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 1–5, Denver, Colorado, June. Associa… our card
  48. Kumutha Swampillai and Mark Stevenson. 2011. Extracting relations within and across sentences. In Proceedings of the International Conference Recent Advances in Natural Language Processing 2011, pages 25–32, Hissar, Bulgaria, September. find
  49. RANLP 2011 Organising Committee. find
  50. Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1499–1509, Lisbon, Portugal, S… our card
  51. Kristina Toutanova, Victoria Lin, Wen-tau Yih, Hoifung Poon, and Chris Quirk. 2016. Compositional learning of embeddings for relation paths in knowledge base and text. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1434–1444,… find
  52. Jianxiang Wang and Man Lan. 2015. A refined end-to-end discourse parser. In Proceedings of the Nineteenth Conference on Computational Natural Language Learning - Shared Task, pages 17–24, Beijing, China, July. Association for Computational Linguistics. our card
  53. Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. 2009. Feature find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/quirk-2017-distant-supervision-relation-extraction-beyond/