Elamur Research
☰
home / papers / 2020 / Named Entity Recognition as Dependency Parsing

Named Entity Recognition as Dependency Parsing

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2020.acl-main.577.pdf

Summary

The paper presents a single NER system that handles both nested entities, such as an organisation name containing a country name, and ordinary flat entities. The authors borrow the biaffine scorer from Dozat and Manning's (2017) graph-based dependency parser. Instead of linking heads to dependents, the model pairs every possible start token with every possible end token and assigns each span an entity label or a non-entity label.

Each token is represented by BERT Large, fastText and character-CNN embeddings, which feed a 3-layer BiLSTM. Two separate FFNNs then build start and end representations for the biaffine classifier. At decoding time, candidate spans are ranked by score, and simple rules remove spans whose boundaries overlap inconsistently with better-scored ones. For flat NER, the rules also remove any containment.

Using one hyperparameter setting throughout, the model reached state-of-the-art F1 on eight benchmarks: three nested NER corpora and five flat ones. This shows that a structured, span-based parser-style approach can outperform sequence labelling. The code is released as open source.

Key points

  • Nested NER results (F1):
    • ACE 2004: 86.7, which is +2.0 over the previous best.
    • ACE 2005: 85.4, which is +1.1.
    • GENIA v3.0.2: 80.5, which is +2.2 and the largest gain reported.
  • Flat NER results (F1):
    • ONTONOTES (English, 18 categories): 91.3, which is +2.1.
    • CONLL 2003 English: 93.5, slightly above the earlier best of 93.4.
    • CONLL 2003 German: 86.4, which is +1.3.
    • CONLL 2002 Spanish: 90.3, which is +1.5.
    • CONLL 2002 Dutch: about +1.0.
    • Revised German data: about 2 points above Akbik et al. (2018).
  • Data splits: ACE 2004 and ACE 2005 use the 80/10/10 document splits of Lu and Roth (2015). GENIA uses a 90/10 split, trained for 50 epochs without a development set. CONLL models are trained on train plus development data.
  • Ablation (ONTONOTES development set, full model 89.9 F1):
    • Removing BERT costs 2.4 points.
    • Replacing the biaffine scorer with a CRF costs 0.8.
    • Removing fastText costs 0.4.
    • Removing character embeddings costs 0.1.
  • Resources: code is available at github.com/juntaoy/biaffine-ner. Main settings are BiLSTM size 200, FFNN size 150, and Adam with learning rate 1e-3.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition · Dependency parsing

Cited by

136 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 133 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.577

@inproceedings{yu2020named,
  title = {{Named Entity Recognition as Dependency Parsing}},
  author = {Yu, Juntao and Bohnet, Bernd and Poesio, Massimo},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.577},
  url = {https://doi.org/10.18653/v1/2020.acl-main.577}
}

References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alan Akbik, Tanja Bergmann, and Roland Vollgraf. 2019. Pooled contextualized embeddings for named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), p… find
  2. Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018. Contextual string embeddings for sequence labeling. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1638–1649, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find our card
  3. Barry Haddow Beatrice Alex and Claire Grover. 2007. Recognising nested named entities in biomedical text. In Proc. of BioNLP, pages 65–72. find
  4. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606. find
  5. Jason PC Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional lstm-cnns. Transactions of the Association for Computational Linguistics, 4:357–370. find our card
  6. NER. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5840–5850, Florence, Italy. Association for Computational Linguistics. find
  7. Meizhi Ju, Makoto Miwa, and Sophia Ananiadou. 2018. A neural layered model for nested named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1446–1459,… find
  8. Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 673–677, Florence, Italy. Association for Computational Linguistics. find
  9. Arzoo Katiyar and Claire Cardie. 2018. Nested named entity recognition revisited. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 861–871, New Orleans, Louisiana. Associat… find
  10. J.-D. Kim, T. Ohta, Y. Tateisi, and J. Tsujii. 2003. GE-NIA corpus—a semantically annotated corpus for bio-textmining. Bioinformatics, 19(suppl 1 ) : i180− −i182. find
  11. Kevin Clark, Minh-Thang Luong, Christopher D. Manning, and Quoc Le. 2018. Semi-supervised sequence modeling with cross-view training. In Proceedings of the 2018 Conference on Empirical Meth- Guillaume Lample, Miguel Ballesteros, Sandeep Subra-ods in Natural Language Processing, pages 1914– manian, K… find our card
  12. Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019. A gen-Jenny Rose Finkel and Christopher D. Manning. 2009. eral framework for information extraction using dy-Nested named entity recognition. In Proceedings of namic span graphs. In Proceedings of the 2019 Con-… find
  13. Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Annual Confer- Changmeng Zheng, Yi Cai, Jingyun Xu, Ho-fung Leung, and Guandong Xu. 2019. A boundary-aware neural ence of the North American Chapter of the Association model for nested named entity recognition. I… find
  14. Dan Shen, Jie Zhang, Guodong Zhou, Jian Su, and Chew-Lim Tan. 2003. Effective adaptation of a Hidden Markov Model-based Named Entity Recognizer for the biomedical domain. In Proceedings of the ACL 2003 Workshop on Natural Language Processing in Biomedicine. find
  15. Mohammad Golam Sohrab and Makoto Miwa. 2018. Deep exhaustive model for nested named entity recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2843–2849, Brussels, Belgium. Association for Computational Linguistics. find
  16. Jana Straková, Milan Straka, and Jan Hajic. 2019. Neural architectures for nested NER through linearization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5326–5331, Florence, Italy. Association for Computational Linguistics. find
  17. Emma Strubell, Patrick Verga, David Belanger, and Andrew McCallum. 2017. Fast and accurate entity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2670–2680, Copenhagen, Denmark. Association for Computati… find
  18. Erik F. Tjong Kim Sang. 2002. Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition. In COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002). find
  19. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings find