Elamur Research⌕
☰
home / papers / 2018 / A Neural Layered Model for Nested Named Entity Recognition

A Neural Layered Model for Nested Named Entity Recognition

Summary

Many named entities contain other entities inside them, such as a protein name within a longer DNA mention. Most NER systems produce only one flat layer of labels, so this finer structure is lost. The authors propose Layered-BiLSTM-CRF, a neural model that finds nested mentions by applying a flat tagger repeatedly, from the innermost entities outward. Each flat NER layer follows Lample et al. (2016) and pairs a bidirectional LSTM with a CRF decoder. When a layer detects entities, the LSTM vectors spanning each one are averaged into a single token-like vector. This vector feeds the next layer, so outer mentions are built from inner ones. All layers share parameters. New layers are added only while entities keep appearing, so the depth adapts to each sentence. The model uses no hand-crafted features or external knowledge. On GENIA and ACE2005 it outperforms earlier feature-based nested NER systems, and it also works as a standard flat tagger on JNLPBA. This shows that end-to-end neural learning can exploit dependencies between inner and outer entities.

Key points

  • The architecture stacks shared BiLSTM-CRF layers dynamically and stops once a layer predicts no further entities. Input words are represented by pretrained word embeddings combined with character-level BiLSTM embeddings.
  • Gold labels follow the BIO scheme, assigned in inside-to-outside order. A given span may receive a given type only once, so each word gets as many labels as the nesting depth.
  • The model reaches 74.7% F-score on GENIA and 72.2% on ACE2005. These are gains of 3.9 and 9.1 percentage points over the previous feature-based state of the art.
  • On the flat JNLPBA corpus it obtains 75.55% F-score, which is comparable to the best reported results.
  • Hyperparameters were tuned with Bayesian optimization. Code is released at github.com/meizhiju/layered-bilstm-crf.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition

Cited by

69 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 63 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1131

@inproceedings{ju2018neural,
  title = {{A Neural Layered Model for Nested Named Entity Recognition}},
  author = {Ju, Meizhi and Miwa, Makoto and Ananiadou, Sophia},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1131},
  url = {https://doi.org/10.18653/v1/n18-1131}
}

References

43 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Beatrice Alex, Barry Haddow, and Claire Grover. 2007. Recognising nested named entities in biomedical text. In Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing. Association for Computational Linguistics, Association for Computational Linguistics… find
  2. Balu Bhasuran, Gurusamy Murugesan, Sabenabanu Abdulkadhar, and Jeyakumar Natarajan. 2016. Stacked ensemble combined with fuzzy matching for biomedical named entity recognition of diseases. Journal of biomedical informatics 64(Supplement C):1–9. https://doi.org/https://doi. org/10.1016/j.jbi.2016.09.… — [doi:10.1016/j.jbi.2016.09.009](https://doi.org/10.1016/j.jbi.2016.09.009) ↗
  3. Kate Byrne. 2007. Nested named entity recognition in historical archive text. In ICSC. IEEE Computer Society, pages 589–596. http://dblp.uni-trier.de/db/conf/ semco/icsc2007.html\#Byrne07. ↗
  4. Mourad Gridach. 2017. Character-level neural network for biomedical named entity recognition. Journal of biomedical informatics 70:85–91. https://doi.org/10.1016/j.jbi.2017.05.002. — [doi:10.1016/j.jbi.2017.05.002](https://doi.org/10.1016/j.jbi.2017.05.002) ↗
  5. Baohua Gu. 2006. Recognizing nested named entities in GENIA corpus. In Proceedings of the HLT-NAACL BioNLP Workshop on Linking Natural Language and Biology. Association for Computational Linguistics, New York, New York, LNLBioNLP ’06, pages 112–113. http://www.aclweb.org/anthology/W/W06/W06-3318. ↗
  6. Nitish Gupta, Sameer Singh, and Dan Roth. 2017. Entity linking via joint encoding of types, descriptions, and context. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pages 2671–2680. https://w… find
  7. Billy Chiu, Gamal Crichton, Anna Korhonen, and Sampo Pyysalo. 2016. How to train good word embeddings for biomedical NLP. Proceedings of the 15th Workshop on Biomedical Natural Language Processing pages 166–174. http://www.aclweb.org/anthology/W16-2922. ↗
  8. Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. CoRR abs/1207.0580. http://arxiv.org/abs/ 1207.0580. ↗
  9. HC Cho, N Okazaki, M Miwa, and J Tsujii. 2010. Nersuite: a named entity recognition toolkit. Tsujii Laboratory, Department of Information Science, University of Tokyo, Tokyo, Japan http://nersuite.nlplab.org/. ↗
  10. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9(8):1735–1780. https://doi.org/10. 1162/neco.1997.9.8.1735. find
  11. Xiaocheng Feng, Lifu Huang, Duyu Tang, Heng Ji, Bing Qin, and Ting Liu. 2016. A language-independent neural network for event detection. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, Ber… find our card
  12. Jenny Rose Finkel and Christopher D Manning. 2009. Nested named entity recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1-Volume 1. Association for Computational Linguistics, Association for Computational Linguistics, Singapore, pages 141… find our card
  13. Pavlina Fragkou. 2017. Applying named entity recognition and co-reference resolution for segmenting english texts. Progress in Artificial Intelligence 6(4):325–346. https://doi.org/10.1007/ s13748-017-0127-3. find
  14. Christoph Goller and Andreas Kuchler. 1996. Learning task-dependent distributed representations by backpropagation through structure. In Neural Networks, 1996., IEEE International Conference on. IEEE, volume 1, pages 347–352. https://doi.org/ 10.1109/ICNN.1996.548916. — [doi:10.1109/icnn.1996.548916](https://doi.org/10.1109/icnn.1996.548916) find
  15. Jin-Dong Kim, Tomoko Ohta, Yuka Tateisi, and Junichi Tsujii. 2003. GENIA corpus a semantically annotated corpus for bio-textmining. Bioinformatics 19(suppl. 1):i180–i182. find
  16. Jin-Dong Kim, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Nigel Collier. 2004. Introduction to the bio-entity recognition task at JNLPBA. In Proceedings of the international joint workshop on natural language processing in biomedicine and its applications. Association for Computational Lingui… find our card
  17. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR abs/1412.6980. https://arxiv.org/abs/ 1412.6980. ↗
  18. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Asso… find
  19. Robert Leaman and Zhiyong Lu. 2016. Taggerone: joint named entity recognition and normalization with semi-markov models. Bioinformatics 32(18):2839–2846. https://doi.org/10. 1093/bioinformatics/btw343. find
  20. Robert Leaman, Chih-Hsuan Wei, and Zhiyong Lu. 2015. tmChem: a high performance approach for chemical named entity recognition and normalization. Journal of cheminformatics 7(1):S3. https://doi.org/10.1186/ 1758-2946-7-S1-S3. find
  21. Fei Li, Meishan Zhang, Bo Tian, Bo Chen, Guohong Fu, and Donghong Ji. 2017. Recognizing irregular entities in biomedical text via deep neural networks. Pattern Recognition Letters 105:105–113. https://doi.org/10.1016/ j.patrec.2017.06.009. find
  22. Xiao Ling and Daniel S. Weld. 2012. Fine-grained entity recognition. In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence. AAAI Press, AAAI’12, pages 94– 100. http://dl.acm.org/citation.cfm? id=2900728.2900742. ↗
  23. Wei Lu and Dan Roth. 2015. Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 857–867. http://aclweb.org/ anthology/D15-110… ↗
  24. Xuezhe Ma and Eduard H. Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf pages 1064–1074. http://www.aclweb.org/ anthology/P16-1101. ↗ our card
  25. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc-Closky. 2014. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Co… find our card
  26. Michał Marcińczuk. 2015. Automatic construction of complex features in conditional random fields for named entities recognition. In Proceedings of the International Conference Recent Advances in Natural Language Processing. IN-COMA Ltd. Shoumen, BULGARIA, Hissar, Bulgaria, pages 413–419. http://www… find
  27. Aldrian Obaja Muis and Wei Lu. 2017. Labeling gaps between words: Recognizing overlapping mentions with mention separators. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pages 2598–2608. http… find
  28. Vardaan Pahuja, Anirban Laha, Shachar Mirkin, Vikas Raykar, Lili Kotlerman, and Guy Lev. 2017. Joint learning of correlated sequence labeling tasks using bidirectional recurrent neural networks pages 548–552. https://doi.org/10.21437/ Interspeech.2017-1247. find
  29. Tim Rocktäschel, Michael Weidlich, and Ulf Leser. 2012. ChemSpot: a hybrid system for chemical named entity recognition. Bioinformatics 28(12):1633–1640. find
  30. Dan Shen, Jie Zhang, Guodong Zhou, Jian Su, and Chew-Lim Tan. 2003. Effective adaptation of a hidden markov model-based named entity recognizer for biomedical domain. In Proceedings of the ACL 2003 workshop on Natural language processing in biomedicine-Volume 13. Association for Computational Lingui… find
  31. Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems. Curran Associates Inc., USA, pages 2951–2959. http://dl.acm.org/citation.cfm?id= 2999325.2999464. ↗
  32. M. Stone and R. Arora. 2017. Identifying nominals with no head match co-references using deep learning. CoRR abs/1710.00936. https://arxiv.org/abs/1710.00936. — [arXiv:1710.00936](https://arxiv.org/abs/1710.00936) ↗
  33. Emma Strubell, Patrick Verga, David Belanger, and Andrew McCallum. 2017. Fast and accurate entity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark,… find our card
  34. Lluı́s Màrquez, Luis Villarejo, M. A. Martı́, and Mariona Taulé. 2007. Semeval-2007 task 09: Multi-level semantic annotation of catalan and spanish. In Proceedings of the 4th International Workshop on Semantic Evaluations. Association for Computational Linguistics, Stroudsburg, PA, USA, SemEval ’0… find
  35. Seiya Tokui, Kenta Oono, Shohei Hido, and Justin Clayton. 2015. Chainer: a next-generation open source framework for deep learning. In Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS). volume 5. find
  36. Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures pages 1105–1116. http://www.aclweb.org/anthology/P16-1105. ↗ our card
  37. Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2006. find
  38. ACE 2005 multilingual training corpus. Linguistic Data Consortium, Philadelphia 57. find
  39. Item Documents Sentences Split percentage DNA RNA Protein Cell Line Cell Type Outermost entity Nested level Entities in level 1 Entities in level 2 Entities in level 3 Entities in level 4 Entity avg. length Multi-token entity Overall entities Yefeng Wang. 2009. Annotating and recognising named entit… find
  40. Paul J Werbos. 1990. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE 78(10):1550–1560. https://doi.org/ 10.1109/5.58337. — [doi:10.1109/5.58337](https://doi.org/10.1109/5.58337) find
  41. Mingbin Xu and Hui Jiang. 2016. A FOFE-based local detection approach for named entity recognition and mention detection. arXiv preprint arXiv:1611.00801 https://arxiv.org/abs/ 1611.00801. — [arXiv:1611.00801](https://arxiv.org/abs/1611.00801) ↗
  42. Jie Zhang, Dan Shen, Guodong Zhou, Jian Su, and Chew-Lim Tan. 2004. Enhancing HMM-based biomedical named entity recognition by studying special phenomena. Journal of biomedical informatics 37(6):411–422. https://doi.org/https://doi.org/ 10.1016/j.jbi.2004.08.005. — [doi:10.1016/j.jbi.2004.08.005](https://doi.org/10.1016/j.jbi.2004.08.005) find
  43. Item Documents Sentences FAC GPE LOC ORG PER VEH WEA Outermost entity Nested level Entities in level 1 Entities in level 2 Entities in level 3 Entities in level 4 Entities in level 5 Entities in level 6 Entity avg. length Multi-token entity Overall entities Data Statistics and Preprocessing find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ju-2018-neural-layered-model-nested-named/