Elamur Research
☰
home / papers / 2018 / Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction

Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D18-1360.pdf

Summary

Scientific literature keeps growing, so researchers need automatic ways to see which tasks, methods and datasets exist and how they connect. Information extraction (IE) in this domain is hard for two reasons. Expert annotation is costly, and many relations span sentence boundaries, often through vague mentions like "it" or "our method". The authors built SciERC, a corpus of 500 AI paper abstracts from the Semantic Scholar Corpus. It is jointly annotated for entities, relations and coreference clusters. They also propose SciIE, a multi-task model that extends span-based coreference and semantic role labeling architectures. SciIE enumerates all candidate spans, so it can handle overlapping spans. It treats entity typing, relation classification and coreference as classification problems over one shared span representation, with no hand-crafted features or pipeline steps. Joint training reduces error propagation between tasks and lets coreference carry information across sentences. SciIE beats earlier scientific IE systems on SciERC and SemEval 17. Applied to a large paper collection, it yields a knowledge graph that is denser and more useful when coreference links are propagated.

Key points

  • SciERC dataset. 500 abstracts from 12 AI venues spanning general AI, NLP, speech, ML and CV. It has 6 entity types (Task, Method, Metric, Material, Other-ScientificTerm, Generic) and 7 relation types, such as Used-for, Part-of and Compare.
  • Annotation scale and agreement.
    • About 8,089 entities, 4,716 relations, 2,752 coreference links and 1,023 clusters.
    • This is 9.4 relations per abstract, against 3.2 for SemEval 18 and 1.3 for SemEval 17.
    • Kappa scores: 76.9% for entities, 67.8% for relations and 63.8% for coreference.
  • Multi-task ablation. Adding coreference improves relation extraction by about 7.1% relative. Adding relation extraction improves coreference by about 4.9% relative.
  • SemEval 17 results. SciIE outperforms feature-based state-of-the-art systems, with the largest gain in span identification. The authors attribute this to span enumeration rather than BIO tagging.
  • Knowledge graph evaluation.
    • Four experts judged about 1.5k relation triples.
    • Precision was above 84% both with and without coreference.
    • With coreference, recall was clearly higher, raising AUC from 0.695 to 0.751.
  • Release. Data and code are released at nlp.cs.washington.edu/sciIE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

260 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 250 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1360 · arXiv 1808.09602

@inproceedings{luan2018multi,
  title = {{Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction}},
  author = {Luan, Yi and He, Luheng and Ostendorf, Mari and Hajishirzi, Hannaneh},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1360},
  eprint = {1808.09602},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d18-1360}
}

References

49 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amjad Abu-Jbara and Dragomir Radev. 2011. Coherent citation-based summarization of scientific papers. In Proc. Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. volume 1, pages 500–509. find
  2. Heike Adel and Hinrich Schütze. 2017. Global normalization of convolutional neural networks for joint entity and relation classification. In Proc. Conf. Empirical Methods Natural Language Process. (EMNLP). pages 1723–1729. find
  3. Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, et al. 2018. Construction of the literature graph in semantic scholar. In Proc. Conf. North American Assoc. for Computational Linguistics: Human Lan… find
  4. Waleed Ammar, Matthew Peters, Chandra Bhagavatula, and Russell Power. 2017. The ai2 system at semeval-2017 task 10 (scienceie): semi-supervised end-to-end entity and relation extraction. In Proc. Int. Workshop on Semantic Evaluation (SemEval). pages 592–596. find
  5. Ashton Anderson, Dan McFarland, and Dan Jurafsky. 2012. Towards a computational history of the ACL: 1980-2008. In Proc. ACL Special Workshop on Rediscovering 50 Years of Discoveries. pages 13–21. \#\# Acknowledgments Awais Athar and Simone Teufel. 2012a. Context-enhanced citation sentiment detection.… find
  6. This research was supported by the Office of Naval Research under the MURI grant N00014-18-1-Awais Athar and Simone Teufel. 2012b. Detection of implicit citations for sentiment detection. In Proc. find
  7. ACL Workshop on Detecting Structure in Scholarly Discourse. pages 18–26. find
  8. Isabelle Augenstein, Mrinal Das, Sebastian Riedel, Lakshmi Vikraman, and Andrew McCallum. 2017. find
  9. Semeval 2017 task 10: ScienceIE - extracting keyphrases and relations from scientific publications. In Proc. Int. Workshop on Semantic Evaluation (SemEval). find
  10. Isabelle Augenstein and Anders Søgaard. 2017. Multi-task learning of keyphrase boundary classification. In Proc. Annu. Meeting Assoc. for Computational Linguistics (ACL). pages 341–346. find
  11. Kevin Clark and Christopher D. Manning. 2016. Improving coreference resolution by learning entity-level distributed representations. CoRR abs/1606.01323. find our card
  12. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proc. Int. Conf. Machine Learning (ICML). pages 160– 167. find
  13. Huy Hoang Nhat Do, Muthu Kumar Chandrasekaran, Philip S Cho, and Min Yen Kan. 2013. Extracting and matching authors and affiliations in scholarly documents. In Proc. ACM/IEEE-CS Joint Conference on Digital libraries. pages 219–228. find
  14. Kata Gábor, Davide Buscaldi, Anne-Kathrin Schumann, Behrang QasemiZadeh, Haı̈fa Zargayouna, and Thierry Charnois. 2018. Semeval-2018 Task 7: Semantic relation extraction and classification in scientific papers. In Proc. Int. Workshop on Semantic Evaluation (SemEval). find
  15. Kata Gabor, Haifa Zargayouna, Davide Buscaldi, Isabelle Tellier, and Thierry Charnois. 2016. Semantic annotation of the ACL anthology corpus for the automatic analysis of scientific literature. In Proc. Language Resources and Evaluation Conference (LREC). find
  16. Kata Gábor, Haı̈fa Zargayouna, Isabelle Tellier, Davide Buscaldi, and Thierry Charnois. 2016. Unsupervised relation extraction in specialized corpora using sequence mining. In International Symposium on Intelligent Data Analysis. Springer, pages 237–248. find
  17. Yarin Gal and Zoubin Ghahramani. 2016. A theoretically grounded application of dropout in recurrent neural networks. In Proc. Annu. Conf. Neural Inform. Process. Syst. (NIPS). find our card
  18. Sonal Gupta and Christopher D Manning. 2011. Analyzing the dynamics of research by extracting key aspects of scientific papers. In Proc. IJCNLP. pages 1–9. find
  19. Luheng He, Kenton Lee, Omer Levy, and Luke Zettlemoyer. 2018. Jointly predicting predicates and arguments in neural semantic role labeling. In ACL. find our card
  20. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735– 1780. find
  21. Kokil Jaidka, Muthu Kumar Chandrasekaran, Beatriz Fisas Elizalde, Rahul Jha, Christopher Jones, Min-Yen Kan, Ankur Khanna, Diego Molla-Aliod, Dragomir R Radev, Francesco Ronzano, et al. 2014. The computational linguistics summarization pilot task. In Proc. Text Analysis Conference. find
  22. Miray Kas. 2011. Structures and statistics of citation networks. Technical report, DTIC Document. find
  23. Arzoo Katiyar and Claire Cardie. 2017. Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. In Proc. Annu. Meeting Assoc. for Computational Linguistics (ACL). volume 1, pages 917–928. find
  24. Sigrid Klerke, Yoav Goldberg, and Anders Søgaard. 2016. Improving sentence compression by learning to predict gaze. In HLT-NAACL. find
  25. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proc. Conf. North American Assoc. for Computational Linguistics (NAACL). find
  26. Kenton Lee, Luheng He, Mike Lewis, and Luke S. Zettlemoyer. 2017. End-to-end neural coreference resolution. In EMNLP. find our card
  27. Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In NAACL. find our card
  28. Yi Luan, Chris Brockett, Bill Dolan, Jianfeng Gao, and Michel Galley. 2017a. Multi-task learning for speaker-role adaptation in neural conversation models. In Proc. IJCNLP. find
  29. Yi Luan, Yangfeng Ji, Hannaneh Hajishirzi, and Boyang Li. 2016. Multiplicative representations for unsupervised semantic role induction. In Proc. Annu. Meeting Assoc. for Computational Linguistics (ACL). page 118. find
  30. Yi Luan, Mari Ostendorf, and Hannaneh Hajishirzi. 2017b. Scientific information extraction with semi-supervised neural tagging. In Proc. Conf. Empirical Methods Natural Language Process. (EMNLP). find
  31. Yi Luan, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. The uwnlp system at semeval-2018 task 7: Neural relation extraction model with selectively incorporated concept embeddings. In Proc. Int. Workshop on Semantic Evaluation (SemEval). pages 788– 792. find
  32. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using lstms on sequences and tree structures. In Proc. Annu. Meeting Assoc. for Computational Linguistics (ACL). pages 1105–1116. find our card
  33. Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih. 2017. Cross-sentence n-ary relation extraction with graph lstms. Trans. Assoc. for Computational Linguistics (TACL) 5:101– 115. find
  34. Matthew Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In Proc. Annu. Meeting Assoc. for Computational Linguistics (ACL). volume 1, pages 1756–1765. find our card
  35. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL. find
  36. Behrang QasemiZadeh and Anne-Kathrin Schumann. 2016. The ACL RD-TEC 2.0: A language resource for evaluating term extraction and entity recognition methods. In LREC. find
  37. Chris Quirk and Hoifung Poon. 2017. Distant supervision for relation extraction beyond the sentence boundary. In Proc. European Chapter Assoc. for Computational Linguistics (EACL). pages 1171– 1182. find
  38. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In HLT-NAACL. find our card
  39. Yan Xu, Ran Jia, Lili Mou, Ge Li, Yunchuan Chen, Yangyang Lu, and Zhi Jin. 2016. Improved relation classification by deep recurrent neural networks with data augmentation. In Proc. Int. Conf. Computational Linguistics (COLING). pages 1461–1470. find
  40. Congle Zhang, Stephen Soderland, and Daniel S. Weld. 2015. Exploiting parallel news streams for unsupervised event extraction. TACL 3:117–129. find
  41. Meishan Zhang, Yue Zhang, and Guohong Fu. 2017. End-to-end neural relation extraction with global optimization. In Proc. Conf. Empirical Methods Natural Language Process. (EMNLP). pages 1730–1740. find
  42. Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. 2017. Joint extraction of entities and relations based on a novel tagging scheme. In Proc. Annu. Meeting Assoc. for Computational Linguistics (ACL). volume 1, pages 1227–1236. find
  43. Marek Rei. 2017. Semi-supervised multitask learning for sequence labeling. In Proc. Annu. Meeting Assoc. for Computational Linguistics (ACL). find
  44. Yanchuan Sim, Noah A Smith, and David A Smith. 2012. Discovering factions in the computational linguistics community. In Proc. ACL Special Workshop on Rediscovering 50 Years of Discoveries. pages 22– 32. find
  45. Sameer Singh, Sebastian Riedel, Brian Martin, Jiaping Zheng, and Andrew McCallum. 2013. Joint inference of entities, relations, and coreference. In Proc. of the 2013 workshop on Automated knowledge base construction. ACM, pages 1–6. find
  46. Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun’ichi Tsujii. 2012. Brat: a web-based tool for nlp-assisted text annotation. In Proc. European Chapter Assoc. for Computational Linguistics (EACL). pages 102– 107. find
  47. Swabha Swayamdipta, Sam Thomson, Chris Dyer, and Noah A. Smith. 2017. Frame-semantic parsing with softmax-margin segmental rnns and a syntactic scaffold. CoRR abs/1706.09528. find our card
  48. Chen-Tse Tsai, Gourab Kundu, and Dan Roth. 2013. Concept-based analysis of scientific literature. In Proc. ACM Int. Conference on Information & Knowledge Management. ACM, pages 1733–1738. find
  49. Adam Vogel and Dan Jurafsky. 2012. He said, she said: Gender in the ACL anthology. In Proc. ACL Special Workshop on Rediscovering 50 Years of Discoveries. pages 33–41. find