Elamur Research
☰
home / papers / 2020 / A Frustratingly Easy Approach for Entity and Relation Extraction

A Frustratingly Easy Approach for Entity and Relation Extraction

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2010.12812

Summary

End-to-end relation extraction requires finding named entities in text and labeling the relations between them. Recent systems usually handle both steps together, through joint structured decoding or shared encoders, on the assumption that this reduces error propagation. Zhong and Chen revisit a plain pipeline instead. Their system, PURE, trains two separate pre-trained encoders (e.g., BERT, SciBERT, ALBERT). The entity model classifies every candidate span using its boundary token vectors plus a width embedding. The relation model then re-encodes the sentence for each candidate pair. Typed marker tokens are inserted around the subject and object, so entity boundaries and types enter at the input layer. Neighbouring sentences supply extra context. On ACE04, ACE05 and SciERC, this setup beats all earlier joint models that use the same encoders. Ablations show three things: the two tasks benefit from separate representations, early fusion of entity information is key, and cross-sentence context helps. A batched approximation makes inference much cheaper. The work questions whether joint modeling is really needed and offers a strong, simple baseline.

Key points

  • PURE reaches new state-of-the-art results on ACE04, ACE05 and SciERC, with relation F1 gains of 1.7–2.8 points (absolute) over prior joint models using identical encoders. Code and models are released at github.com/princeton-nlp/PURE.
  • Sharing one encoder between the entity and relation tasks lowered both entity and relation F1. An auxiliary relation loss improved entity F1 by less than 0.1% on the ACE05 dev set.
  • The approximate relation model batches many entity pairs in a single pass. It runs 8–16× faster, at a cost of about 1.0 F1 on ACE05.
  • Attempts to counter error propagation did not help significantly. Neither 10-way jackknifing of training entities nor feeding top-scoring spans (λ = 0.4 × sentence length, 96.7% gold-relation recall on ACE05 dev) improved results, and jackknifing actually hurt.
  • The chosen context window sizes are W = 300 in the entity model and W = 100 in the relation model. Relation F1 stopped improving beyond W = 100.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

151 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 148 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2010.12812 · arXiv 2010.12812

@misc{zhong2020frustratingly,
  title = {{A Frustratingly Easy Approach for Entity and Relation Extraction}},
  author = {Zhong, Zexuan and Chen, Danqi},
  year = {2020},
  doi = {10.48550/arxiv.2010.12812},
  eprint = {2010.12812},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2010.12812}
}

References

37 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Željko Agić and Natalie Schluter. 2017. How (not) to train a dependency parser: The curious case of jackknifing part-of-speech taggers. In Association for Computational Linguistics (ACL), pages 679–684. find
  2. Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Adversarial training for multi-context joint entity and relation extraction. In Empirical Methods in Natural Language Processing (EMNLP), pages 2830–2836. find
  3. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. Scibert: A pretrained language model for scientific text. In Empirical Methods in Natural Language Processing (EMNLP), pages 3606–3611. find our card
  4. Razvan Bunescu and Raymond Mooney. 2005. A shortest path dependency kernel for relation extraction. In Empirical Methods in Natural Language Processing (EMNLP), pages 724–731. find
  5. Yee Seng Chan and Dan Roth. 2011. Exploiting syntactico-semantic structures for relation extraction. In Association for Computational Linguistics: Human Language Technologies (ACL-HLT), pages 551–560. find
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics (NAACL), pages 4171–4186. find
  7. Kalpit Dixit and Yaser Al-Onaizan. 2019. Span-level model for relation extraction. In Association for Computational Linguistics (ACL), pages 5308– 5314. find
  8. Radu Florian, Hany Hassan, Abraham Ittycheriah, Hongyan Jing, Nanda Kambhatla, Xiaoqiang Luo, H Nicolov, and Salim Roukos. 2004. A statistical model for multilingual entity detection and tracking. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  9. Radu Florian, Hongyan Jing, Nanda Kambhatla, and Imed Zitouni. 2006. Factorizing complex models: A case study in mention detection. In Association for Computational Linguistics (ACL), pages 473–480. find
  10. Tsu-Jui Fu, Peng-Hsuan Li, and Wei-Yun Ma. 2019. GraphRel: Modeling text as relational graphs for joint entity and relation extraction. In Association for Computational Linguistics (ACL), pages 1409– 1418. find
  11. Luheng He, Kenton Lee, Omer Levy, and Luke Zettlemoyer. 2018. Jointly predicting predicates and arguments in neural semantic role labeling. In Association for Computational Linguistics (ACL), pages 364–369. find our card
  12. Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for information extraction. In Association for Computational Linguistics (ACL), pages 178–181. find
  13. Arzoo Katiyar and Claire Cardie. 2017. Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. In Association for Computational Linguistics (ACL), pages 917–928. find
  14. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations (ICLR). find
  15. Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Empirical Methods in Natural Language Processing (EMNLP), pages 188–197. find our card
  16. Qi Li and Heng Ji. 2014. Incremental joint extraction of entity mentions and relations. In Association for Computational Linguistics (ACL), pages 402–412. find
  17. Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. In Association for Computational Linguistics (ACL), pages 1340–1350. find
  18. Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020. A joint neural model for information extraction with global features. In Association for Computational Linguistics (ACL). find our card
  19. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Empirical Methods in Natural Language Processing (EMNLP), pages 3219–3232. find our card
  20. Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019. A general framework for information extraction using dynamic span graphs. In North American Chapter of the Association for Computational Linguistics (NAACL), pages 3036–3046. find our card
  21. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Association for Computational Linguistics (ACL), pages 1105–1116. find our card
  22. Makoto Miwa and Yutaka Sasaki. 2014. Modeling joint entity and relation extraction with table representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1858–1869. find
  23. Matthew E Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A Smith. 2019. Knowledge enhanced contextual word representations. In Empirical Methods in Natural Language Processing (EMNLP), pages 43–54. find
  24. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Computational Natural Language Learning (CoNLL), pages 147–155. find our card
  25. Erik Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Computational Natural Language Learning (CoNLL), pages 142–147. find
  26. Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. In Association for Computational Linguistics (ACL), pages 2895–2905. find
  27. Changzhi Sun, Yeyun Gong, Yuanbin Wu, Ming Gong, Daxin Jiang, Man Lan, Shiliang Sun, and Nan Duan. 2019. Joint type inference on entities and relations via graph convolutional networks. In Association for Computational Linguistics (ACL), pages 1361– 1370. find
  28. Bruno Taillé, Vincent Guigue, Geoffrey Scoutheeten, and Patrick Gallinari. 2020. Let’s stop incorrect comparisons in end-to-end relation extraction! In Empirical Methods in Natural Language Processing (EMNLP). find
  29. David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. In Empirical Methods in Natural Language Processing (EMNLP), pages 5788–5793. find our card
  30. Jue Wang and Wei Lu. 2020. Two are better than one: Joint entity and relation extraction with table-sequence encoders. In Empirical Methods in Natural Language Processing (EMNLP). find
  31. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… find our card
  32. Dmitry Zelenko, Chinatsu Aone, and Anthony Richardella. 2002. Kernel methods for relation extraction. In Empirical Methods in Natural Language Processing (EMNLP), pages 71–78. find
  33. Meishan Zhang, Yue Zhang, and Guohong Fu. 2017a. End-to-end neural relation extraction with global optimization. In Empirical Methods in Natural Language Processing (EMNLP), pages 1730–1740. find
  34. Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D. Manning. 2017b. Position-aware attention and supervised data improve slot filling. In Empirical Methods in Natural Language Processing (EMNLP), pages 35–45. find
  35. Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: Enhanced language representation with informative entities. In Association for Computational Linguistics (ACL), pages 1441–1451. find
  36. Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. 2017. Joint extraction of entities and relations based on a novel tagging scheme. In Association for Computational Linguistics (ACL), pages 1227–1236. find
  37. GuoDong Zhou, Jian Su, Jie Zhang, and Min Zhang. 2005. Exploring various knowledge in relation extraction. In Association for Computational Linguistics (ACL), pages 427–434. find