Elamur Research⌕
☰
home / papers / 2017 / Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme

Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme

Summary

The paper addresses joint extraction of entities and relations: finding entity pairs in raw text together with a relation type from a fixed inventory. Pipelined systems handle NER and relation classification separately, so errors pass from one stage to the next. Earlier joint systems either depend on heavy feature engineering and external NLP tools or still predict entities and relations as separate outputs. The authors instead treat the whole task as sequence labeling. Each token's tag combines three things: its position inside an entity, a relation type, and the entity's role in the triplet. A tag sequence can then be decoded straight into (Entity1, RelationType, Entity2) triplets. On top of this scheme they compare several LSTM-based end-to-end taggers and propose LSTM-LSTM-Bias, which uses a Bi-LSTM encoder, an LSTM decoder, and a training loss that weights relation-bearing tags more heavily. Experiments use a public distantly supervised dataset released with CoType (Ren et al., 2017). Tagging-based methods outperform most pipelined and joint baselines, and LSTM-LSTM-Bias gives the best overall results. The work matters because it shows that triplet extraction can be framed as ordinary tagging, which avoids hand-built features.

Key points

  • Tag structure. A tag is either "O" or a combination of three parts:

    • a BIES position marker,
    • a relation type,
    • a role number ("1" for the first entity, "2" for the second).

    The tag set size is therefore 2·4·|R| + 1, where |R| is the number of relation types.

  • Decoding tags into triplets. Entities that carry the same relation type are paired into a triplet. When several pairs share a type, entities are matched by proximity. Overlapping relations, where one entity appears in several triplets, are explicitly left out of scope.

  • Models compared.

    • LSTM-CRF
    • LSTM-LSTM
    • LSTM-LSTM-Bias

    The LSTM decoder slightly beats the CRF decoder, which the authors attribute to related tags often lying far apart in the sentence.

  • Bias parameter α. α was varied from 1 to 20. Large values hurt precision and small values hurt recall. The best F1 was reached at α = 10.

  • Error analysis.

    • LSTM-LSTM-Bias leaves the fewest unpaired single entities.
    • Case studies show remaining failures: misleading surface patterns, and entity pairs found correctly but with their roles swapped.

Interesting ideas

  • I1. Encoding a structured output as per-token labels. Packing relation type and argument role into the tag lets a standard sequence tagger produce relational output. The same trick could help with other structured extraction tasks, such as trade-document fields or geographic relations, without designing a new architecture.
  • I2. Class-weighted loss for rare informative tags. Raising the loss weight on non-"O" tags counters the dominance of the "O" class. The single α knob gives explicit control over the precision–recall trade-off, and the same idea suits any sparse tagging task.
  • I3. Checking whether outputs are complete. Measuring the share of predicted entities that never found a partner is a cheap diagnostic. It shows whether a joint model actually connects arguments, beyond what P/R/F reports.
  • I4. Using distant supervision to get training data. Training on an automatically labeled, distantly supervised dataset avoids costly manual annotation. This is relevant for lower-resource languages such as Ukrainian, where annotated relation corpora are scarce.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

115 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 108 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p17-1113

@inproceedings{zheng2017joint,
  title = {{Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme}},
  author = {Zheng, Suncong and Wang, Feng and Bao, Hongyun and Hao, Yuexing and Zhou, Peng and Xu, Bo},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1113},
  url = {https://doi.org/10.18653/v1/p17-1113}
}

References

34 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michele Banko, Michael J Cafarella, Stephen Soderland, Matthew Broadhead, and Oren Etzioni. 2007. Open information extraction from the web. In IJ-CAI. volume 7, pages 2670–2676. find our card
  2. Jason PC Chiu and Eric Nichols. 2015. Named entity recognition with bidirectional lstm-cnns. In Processings of Transactions of the Association for Computational Linguistics. find our card
  3. Cıcero Nogueira et al. dos Santos. 2015. Classifying relations by ranking with convolutional neural networks. In Proceedings of the 53th ACL international conference. volume 1, pages 626–634. find our card
  4. Matthew R Gormley, Mo Yu, and Mark Dredze. 2015. Improved relation extraction with feature-rich compositional embedding models. In Proceedings of the EMNLP. find
  5. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. find
  6. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics. Association for Computational Lingui… find
  7. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991 . find our card
  8. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In EMNLP. volume 3, page 413. find our card
  9. Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of the 43th ACL international conference. page 22. find
  10. Arzoo Katiyar and Claire Cardie. 2016. Investigating lstms for joint extraction of opinion entities and relations. In Proceedings of the 54th ACL international conference. find
  11. John Lafferty, Andrew McCallum, Fernando Pereira, et al. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the eighteenth international conference on machine learning, ICML. volume 1, pages 282–289. find
  12. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the NAACL international conference. find
  13. Qi Li and Heng Ji. 2014. Incremental joint extraction of entity mentions and relations. In Proceedings of the 52rd Annual Meeting of the Association for Computational Linguistics. pages 402–412. find our card
  14. Gang Luo, Xiaojiang Huang, Chin-Yew Lin, and Zaiqing Nie. 2015. Joint entity recognition and disambiguation. In Conference on Empirical Methods in Natural Language Processing. pages 879–888. find
  15. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. pages 3111–3119. find
  16. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL. Association for Computational Linguistics, pages 1003–1011. find
  17. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using lstms on sequences and tree structures. In Proceedings of the 54rd Annual Meeting of the Association for Computational Linguistics. find our card
  18. Makoto Miwa and Yutaka Sasaki. 2014. Modeling joint entity and relation extraction with table representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. pages 1858–1869. find our card
  19. Xiaofeng Yu and Wai Lam. 2010. Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach. In Proceedings of the 21th COLING international conference. pages 1399–1407. find
  20. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes 30(1):3–26. find
  21. Daojian et al. Zeng. 2014. Relation classification via convolutional deep neural network. In Proceedings of the 25th COLING international conference. pages 2335–2344. find our card
  22. Alexandre Passos, Vineet Kumar, and Andrew McCallum. 2014. Lexicon infused phrase embeddings for named entity resolution. In International Conference on Computational Linguistics. pages 78–86. find our card
  23. Xiang Ren, Zeqiu Wu, Wenqi He, Meng Qu, Clare R Voss, Heng Ji, Tarek F Abdelzaher, and Jiawei Han. 2017. Cotype: Joint extraction of typed entities and relations with knowledge bases. In Proceedings of the 26th WWW international conference. find
  24. Feifei Zhai, Saloni Potdar, Bing Xiang, and Bowen Zhou. 2017. Neural models for sequence chunking. In Proceedings of the AAAI international conference. find
  25. Suncong Zheng, Jiaming Xu, Peng Zhou, Hongyun Bao, Zhenyu Qi, and Bo Xu. 2016. A neural network framework for relation extraction: Learning entity semantic and relation pattern. Knowledge-Based Systems 114:12–23. find
  26. Bryan et al. Rink. 2010. Utd: Classifying semantic relations by combining lexical and semantic resources. In Proceedings of the 5th International Workshop on Semantic Evaluation. pages 256–259. find
  27. Sameer Singh, Sebastian Riedel, Brian Martin, Jiaping Zheng, and Andrew McCallum. 2013. Joint inference of entities, relations, and coreference. In Proceedings of the 2013 workshop on Automated knowledge base construction. ACM, pages 1–6. find
  28. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems. pages 3104–3112. find
  29. Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. ACM, pages 1067–1077. find
  30. Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-rmsprop. In COURSERA: Neural networks for machine learning. find
  31. Ashish Vaswani, Yonatan Bisk, Kenji Sagae, and Ryan Musa. 2016. Supertagging with lstms. In Proceedings of the NAACL international conference. pages 232–237. find
  32. Kun et al. Xu. 2015a. Semantic relation classification via convolutional neural networks with simple negative sampling. In Proceedings of the EMNLP. find
  33. Yan et al. Xu. 2015b. Classifying relations via long short term memory networks along shortest dependency paths. In Proceedings of EMNLP international conference. find our card
  34. Bishan Yang and Claire Cardie. 2013. Joint inference for fine-grained opinion extraction. In Proceedings of the 51rd Annual Meeting of the Association for Computational Linguistics. pages 1640–1649. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2017-joint-extraction-entities-relations-based/