Elamur Research⌕
☰
home / papers / 2018 / Nested Named Entity Recognition Revisited

Nested Named Entity Recognition Revisited

Paper · 2018 · Arzoo Katiyar and Claire Cardie

Summary

Standard named entity recognition and mention detection systems tag each token with a single label, so they cannot detect entities embedded inside other entities. Such nesting is frequent: about 17% of GENIA entities sit inside another entity, and roughly 30% of ACE sentences contain nested mentions. Earlier solutions for nesting, such as cascaded CRFs, constituency-parser models and mention hypergraphs or multigraphs, depend on hand-engineered features. Some are also slow, with the parser approach being cubic in sentence length.

Katiyar and Cardie merge the overlapping BILOU tag sequences at each token into one directed hypergraph. An LSTM-based tagger then builds this structure greedily, predicting at every token which outgoing edges and hyperarcs exist. Entities are recovered afterwards by depth-first search over the predicted edges. Inference is linear in sentence length and in the number of possible labels per token, and the model can also be extended to identify each mention's head word.

On ACE2004, ACE2005 and GENIA, the model beats prior feature-based state-of-the-art systems. This shows that neural sequence models can handle nested entities without manual feature engineering.

Key points

  • Representation. Shared BILOU states across overlapping entity sequences are collapsed into single nodes. Hyperarcs link one tail node to several head nodes, and an "O" node is kept at every step so a new entity can begin. The model can represent the same structures as the methods proposed by Lu and Roth (2015) and by Muis and Lu (2017), but it learns them neurally and greedily.
  • Baselines outperformed.
    • ACE2004 and ACE2005: Lu and Roth's (2015) mention hypergraph and Muis and Lu's (2017) multigraph, on mention detection and on joint mention-plus-head extraction.
    • GENIA: the constituency-parser approach of Finkel and Manning (2009).
  • GENIA setup. The first 90% of sentences were used for training and the last 10% for testing, covering five types (DNA, RNA, protein, cell line, cell type). The model used word vectors trained on PubMed data.
  • Softmax vs. sparsemax. Sparsemax output predicts hyperedges better on ACE and recovers more nested mentions, while both variants perform similarly on GENIA.
  • Error analysis. The model processes sentences independently, so it lacks document context; for example, it mislabels "they" referring to a tank. It also always treats "It" as a non-entity, and some ACE head annotations are inconsistent. The authors suggest adding coreference information and using global training as future directions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Cornell UniversityIthaca · United States89 works · 58,422 citations

📚 Shelves 6 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Biochemistry, Genetics and Molecular Biology · 116 works Molecular Biology · 114 works

🧭 Topics 1 Named entity recognition · 101 works

Cite

Cite as Arzoo Katiyar and Claire Cardie. 2018. Nested Named Entity Recognition Revisited. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1079

@inproceedings{katiyar2018nested,
  title = {{Nested Named Entity Recognition Revisited}},
  author = {Arzoo Katiyar and Claire Cardie},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1079},
  url = {https://doi.org/10.18653/v1/n18-1079}
}

References

25 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Beatrice Alex, Barry Haddow, and Claire Grover. 2007. Recognising nested named entities in biomedical text. In Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing. Association for Computational Linguistics, Stroudsburg, PA, USA, BioNLP ’07, pages 6… find
  2. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Asso… our card
  3. Kai-Wei Chang, Rajhans Samdani, and Dan Roth. 2013. A constrained latent variable model for coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Seattle, Washington, USA, pages 601–612. http://ww… our card
  4. Wei Lu and Dan Roth. 2015. Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 857–867. http://aclweb.org/anthology/D15-1102… ↗ our card
  5. Yejin Choi, Eric Breck, and Claire Cardie. 2006. Joint extraction of entities and relations for opinion recognition. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Sydney, Australia, pages 431–439. http://www.aclw… ↗
  6. Gang Luo, Xiaojiang Huang, Chin-Yew Lin, and Zaiqing Nie. 2015. Joint entity recognition and disambiguation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 879–888. https://aclweb.org/antho… ↗
  7. André F. T. Martins and Ramón F. Astudillo. 2016. From softmax to sparsemax: A sparse model of attention and multi-label classification. In Proceedings of the 33rd International Conference on International Conference on Machine Learning -Volume 48. JMLR.org, ICML’16, pages 1614–1623. http://dl.acm… ↗
  8. Jenny Rose Finkel and Christopher D. Manning. 2009. Nested named entity recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Singapore, pages 141– 150. http://www.aclweb.org/anthology/D/D09/D09-1015. ↗ our card
  9. Ryan McDonald and Fernando Pereira. 2005. Identifying gene and protein mentions in text using conditional random fields. BMC Bioinformatics 6(1):S6. https://doi.org/10.1186/1471-2105-6-S1-S6. — [doi:10.1186/1471-2105-6-s1-s6](https://doi.org/10.1186/1471-2105-6-s1-s6) ↗
  10. R Florian, H Hassan, A Ittycheriah, H Jing, N Kambhatla, X Luo, N Nicolov, and S Roukos. 2004. A statistical model for multilingual entity detection and tracking. In Daniel Marcu Susan Dumais and Salim Roukos, editors, HLT-NAACL 2004: Main Proceedings. Association for Computational Linguistics, Bost… our card
  11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, Curra… our card
  12. Arzoo Katiyar and Claire Cardie. 2016. Investigating lstms for joint extraction of opinion entities and relations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. http://aclweb.org/anthol… ↗
  13. Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Assoc… find
  14. Dan Klein and Christopher D. Manning. 2001. Parsing and hypergraphs. In Proceedings of the Seventh International Workshop on Parsing Technologies (IWPT-2001), 17-19 October 2001, Beijing, China. find
  15. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: ger. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, Stroudsburg, PA, USA, ACL ’02, pages 473–480. https://doi.org/10.311… our card
  16. Aldrian Obaja Muis and Wei Lu. 2017. Labeling gaps between words: Recognizing overlapping mentions with mention separators. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pages 2598– 2608. htt… find
  17. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Linguisticae Investigationes 30(1):3–26. Publisher: John Benjamins Publishing Company. find
  18. Tomoko Ohta, Yuka Tateisi, and Jin-Dong Kim. 2002. The genia corpus: An annotated research abstract corpus in molecular biology domain. In Proceedings of the Second International Conference on Human Language Technology Research. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, HLT ’02, pages… find
  19. S Pyysalo, F Ginter, H Moen, T Salakoski, and S Ananiadou. 2013. Distributional Semantics Resources for Biomedical Text Processing, pages 39–44. find
  20. Sunita Sarawagi and William W Cohen. 2005. Semi-markov conditional random fields for information extraction. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, MIT Press, pages 1185–1192. http://papers.nips.cc/paper/2648-semi-markov-conditional-ran… ↗
  21. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15:1929–1958. http://jmlr.org/papers/v15/srivastava14a.html. ↗
  22. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems. MIT Press, Cambridge, MA, USA, NIPS’14, pages 3104–3112. http://dl.acm.org/citation.cfm?id=2969033.29… ↗
  23. Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2. MIT Press, Cambridge, MA, USA, NIPS’15, pages 2773–2781. http://dl.… ↗
  24. Matthew D. Zeiler. 2012. ADADELTA: an adaptive learning rate method. CoRR abs/1212.5701. http://arxiv.org/abs/1212.5701. ↗ our card
  25. GuoDong Zhou and Jian Su. 2002. Named entity recognition using an hmm-based chunk tag- find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/katiyar-2018-nested-named-entity-recognition-revisited/