Elamur Research
☰
home / papers / 2020 / A Joint Neural Model for Information Extraction with Global Features

A Joint Neural Model for Information Extraction with Global Features

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.acl-main.713.pdf

Summary

Joint neural systems for information extraction usually assign each label (entity type, relation, trigger, argument role) with its own local classifier. They ignore how these decisions constrain one another. The authors present ONEIE, which treats a sentence's output as a single information graph. Entity mentions and event triggers are nodes, and relations and argument links are edges. The pipeline works in four steps:

  • a pre-trained BERT encoder builds word vectors, enriched with its third-to-last layer;
  • CRF taggers with BIO tags locate node spans;
  • feed-forward networks score labels for nodes and node pairs;
  • a beam-search decoder selects the graph with the best overall score.

That score includes learned weights on global features. These capture cross-subtask patterns, such as a GPE being an implausible VICTIM of a DIE event, and cross-instance patterns, such as an ELECT event rarely having two PERSON arguments. Adding these features improves results, and ONEIE matches or exceeds the previous end-to-end state of the art (Wadden et al., 2019) on all subtasks. Because it uses no language-specific features, the approach transfers to new languages and supports multilingual training.

Key points

  • The framework has four stages: encoding, identification, classification, and decoding. Entities and triggers get separate taggers, but the taggers' type predictions are discarded. Final types are chosen jointly during decoding to reduce error propagation.
  • Global features fall into 11 categories (Table 1), covering argument roles, relations, and triggers. Their weights are learned during training, and the features are interpretable.
  • ONEIE builds on Wadden et al. (2019), which uses BERT and dynamic span graphs. It replaces span enumeration with CRF tagging, so mention length is not capped by a maximum span width. Unlike Li et al. (2014), it needs no external tools such as dependency parsers.
  • The authors have made their source code and trained English, Spanish, and Chinese models publicly available at blender.cs.illinois.edu/software/oneie.
  • The error analysis identifies several remaining weaknesses:
    • rare (long-tail) words;
    • triggers that fit several event types, e.g., "killed" as ATTACK vs. DIE;
    • errors that deeper syntactic analysis might fix;
    • planned or metaphorical events;
    • inconsistent annotation in the ACE data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Part-of-speech tagging

Cited by

217 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 211 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.713

@inproceedings{lin2020joint,
  title = {{A Joint Neural Model for Information Extraction with Global Features}},
  author = {Lin, Ying and Ji, Heng and Huang, Fei and Wu, Lingfei},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.713},
  url = {https://doi.org/10.18653/v1/2020.acl-main.713}
}

References

26 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Adversarial training for multi-context joint entity and relation extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. find
  2. Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Proces… find
  3. Jason Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association of Computational Linguistics (TACL). find our card
  4. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. find
  5. Natural language processing (almost) from scratch. Journal of Machine Learning Research. find our card
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  7. Greg Durrett and Dan Klein. 2014. A joint model for entity analysis: Coreference, typing, and linking. In Transactions of the Association for Computational Linguistics (TACL). find
  8. Tsu-Jui Fu, Peng-Hsuan Li, and Wei-Yun Ma. 2019. GraphRel: Modeling text as relational graphs for joint entity and relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL2019). find
  9. Rujun Han, Qiang Ning, and Nanyun Peng. 2019. Joint event and temporal relation extraction with shared representations and structured prediction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language P… find
  10. Heng Ji and Ralph Grishman. 2005. Improving name tagging by reference resolution and relation detection. In In Proceedings of ACL 05, Ann Arbor, USA. find
  11. Heng Ji, David Westbrook, and Ralph Grishman. 2005. Using semantic relations to refine coreference decisions. In In Proceedings of HLT/EMNLP 05, Vancouver, B.C., Canada. find
  12. Arzoo Katiyar and Claire Cardie. 2017. Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL2017). find
  13. Johannes Kirschnick, Holmer Hemsen, and Volker Markl. 2016. JEDI: Joint entity and relation detection using type inference. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics System Demonstrations (ACL2016). find
  14. Qi Li, Sam Anzaroot, Wen-Pin Lin, Xiang Li, and Heng Ji. 2011. Joint inference for cross-document information extraction. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (CIKM2011). find
  15. Qi Li, Heng Ji, HONG Yu, and Sujian Li. 2014. Constructing information networks using one single model. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP2014). find
  16. Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL2016). find
  17. Wei Lu and Dan Roth. 2015. Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP2015). find
  18. Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019. A general framework for information extraction using dynamic span graphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tec… find our card
  19. Makoto Miwa and Yutaka Sasaki. 2014. Modeling joint entity and relation extraction with table representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP2014). find
  20. Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL2004). find our card
  21. Avirup Sil and Alexander Yates. 2013. Re-ranking for joint named-entity recognition and linking. In Proceedings of the 22nd ACM international conference on Conference on Information & Knowledge Management (CIKM2013). find
  22. Changzhi Sun, Yeyun Gong, Yuanbin Wu, Ming Gong, Daxin Jiang, Man Lan, Shiliang Sun, and Nan Duan. 2019. Joint type inference on entities and relations via graph convolutional networks. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL2019). find
  23. David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langua… find our card
  24. Bishan Yang and Tom M. Mitchell. 2016. Joint extraction of events and entities within a document context. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT2016). find
  25. Tongtao Zhang, Heng Ji, and Avirup Sil. 2019. Joint entity and event extraction with generative adversarial imitation learning. Data Intelligence. find
  26. Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. 2017. Joint extraction of entities and relations based on a novel tagging scheme. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL2017). find