Elamur Research⌕
☰
home / papers / 2018 / A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

Summary

This survey asks how deep learning has changed named entity recognition (NER) compared with older approaches. The authors searched Google, Google Scholar and Semantic Scholar, then traced each architecture back to the paper that first introduced it. They group systems into four families: knowledge-based (lexicons, rules), unsupervised/bootstrapped, feature-engineered supervised learners (CRF, SVM, MaxEnt, AdaBoost) and neural models. The neural models are further divided by input representation: word, character, word+character, and word+character+affix. The survey also describes the main shared-task datasets and the evaluation metrics: MUC scoring, CoNLL exact match, and relaxed vs. strict F1. Results are compared on CoNLL 2002/2003 in four languages and on the biomedical DrugNER set. The comparison shows three things:

  • Neural systems that learn their own features beat hand-engineered systems, even when the latter use gazetteers.
  • Hybrid word+character networks are usually the strongest design.
  • Bringing old feature ideas into neural models still helps: the authors' affix-augmented model sets new top scores.

This makes the paper a practical map for choosing an NER architecture, including for low-resource and Slavic-language work.

Key points

  • 154 papers were screened and 83 were kept for the survey.
  • On CoNLL news data, neural models without external resources beat the best feature-engineered results by 1.59% (Spanish), 2.34% (German), 0.36% (English) and 0.14% (Dutch).
  • Hybrid models win on English: Chiu and Nichols (2015) beats the best word model (Huang et al., 2015) by 0.52% and the best character model (Kuru et al., 2016, CharNER) by 5.12%.
  • On DrugNER (SemEval 2013 Task 9), a word+character network outperforms the best feature-engineered system by 8.90% on MedLine and 3.50% on the full test set.
  • Yadav et al. (2018) add affix features to Lample et al. (2016). This sets a new state of the art for Spanish, Dutch and German and comes within 1% of the best English model. The listed datasets also include the BSNLP 2017 Slavic task, which covers Ukrainian.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Named entity recognition

Cited by

101 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 98 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{yadav2018survey,
  title = {{A Survey on Recent Advances in Named Entity Recognition from Deep Learning models}},
  author = {Yadav, Vikas and Bethard, Steven},
  booktitle = {Proceedings of the 27th International Conference on Computational Linguistics},
  year = {2018},
  url = {https://aclanthology.org/C18-1182/}
}

References

82 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rodrigo Agerri and German Rigau. 2016. Robust multilingual named entity recognition with shallow semi-supervised features. Artificial Intelligence, 238:63–82. find
  2. Enrique Alfonseca and Suresh Manandhar. 2002. An unsupervised method for general named entity recognition and automated concept discovery. In Proceedings of the 1st international conference on general WordNet, Mysore, India, pages 34–43. find
  3. Rie Kubota Ando and Tong Zhang. 2005a. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(Nov):1817–1853. find
  4. Rie Kubota Ando and Tong Zhang. 2005b. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6(Nov):1817–1853. find
  5. Timothy Baldwin, Marie-Catherine de Marneffe, Bo Han, Young-Bum Kim, Alan Ritter, and Wei Xu. 2015. Shared tasks of the 2015 workshop on noisy user-generated text: Twitter lexical normalization and named entity recognition. In Proceedings of the Workshop on Noisy User-generated Text, pages 126–135. find our card
  6. Darina Benikova, Chris Biemann, and Marc Reznicek. 2014. Nosta-d named entity annotation for german: Guidelines and dataset. In LREC, pages 2524–2531. find
  7. Akash Bharadwaj, David R. Mortensen, Chris Dyer, and Carlos de Juan Carbonell. 2016. Phonologically aware neural model for named entity recognition in low resource transfer settings. In EMNLP. find
  8. Daniel M Bikel, Scott Miller, Richard Schwartz, and Ralph Weischedel. 1997. Nymble: a high-performance learning name-finder. In Proceedings of the fifth conference on Applied natural language processing, pages 194–201. Association for Computational Linguistics. find our card
  9. Robert Bossy, Wiktoria Golik, Zorana Ratkovic, Philippe Bessières, and Claire Nédellec. 2013. Bionlp shared task 2013–an overview of the bacteria biotope task. In Proceedings of the BioNLP Shared Task 2013 Workshop, pages 161–169. find
  10. Xavier Carreras, Lluı́s Màrquez, and Lluı́s Padró. 2002. Named entity extraction using adaboost, proceedings of the 6th conference on natural language learning. August, 31:1–4. find
  11. Raghavendra Chalapathy, Ehsan Zare Borzeshi, and Massimo Piccardi. 2016. An investigation of recurrent neural architectures for drug name recognition. arXiv preprint arXiv:1609.07585. find
  12. Nancy Chinchor and Patricia Robinson. 1997. Muc-7 named entity task definition. In Proceedings of the 7th Conference on Message Understanding, volume 29. find
  13. Jason PC Chiu and Eric Nichols. 2015. Named entity recognition with bidirectional lstm-cnns. arXiv preprint arXiv:1511.08308. find our card
  14. Michael Collins and Yoram Singer. 1999. Unsupervised models for named entity classification. In 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. find our card
  15. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160–167. ACM. find
  16. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug):2493– 2537. find our card
  17. Silviu Cucerzan and David Yarowsky. 2002. Language independent ner using a unified model of internal and contextual evidence. In proceedings of the 6th conference on Natural language learning-Volume 20, pages 1–4. Association for Computational Linguistics. find
  18. Louise Delėger, Robert Bossy, Estelle Chaix, Mouhamadou Ba, Arnaud Ferrė, Philippe Bessieres, and Claire Nėdellec. 2016. Overview of the bacteria biotope task at bionlp shared task 2016. In Proceedings of the 4th BioNLP Shared Task Workshop, pages 12–22. find
  19. Franck Dernoncourt, Ji Young Lee, and Peter Szolovits. 2017. Neuroner: an easy-to-use program for named-entity recognition based on neural networks. arXiv preprint arXiv:1705.05487. find
  20. Chuanhai Dong, Jiajun Zhang, Chengqing Zong, Masanori Hattori, and Hui Di. 2016. Character-based lstm-crf with radical-level features for chinese named entity recognition. In Natural Language Understanding and Intelligent Applications, pages 239–250. Springer. find
  21. Safaa Eltyeb and Naomie Salim. 2014. Chemical named entities recognition: a review on approaches and applications. Journal of cheminformatics, 6(1):17. find
  22. Wael Etaiwi, Arafat Awajan, and Dima Suleiman. 2017. Statistical arabic name entity recognition approaches: A survey. Procedia Computer Science, 113:57–64. find
  23. Oren Etzioni, Michael Cafarella, Doug Downey, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel S Weld, and Alexander Yates. 2005. Unsupervised named-entity extraction from the web: An experimental study. Artificial intelligence, 165(1):91–134. find
  24. G David Forney. 1973. The viterbi algorithm. Proceedings of the IEEE, 61(3):268–278. find
  25. Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2015. Multilingual language processing from bytes. arXiv preprint arXiv:1512.00103. find
  26. Ralph Grishman and Beth Sundheim. 1996. Message understanding conference-6: A brief history. In COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics, volume 1. find
  27. Maryam Habibi, Leon Weber, Mariana Neves, David Luis Wiegandt, and Ulf Leser. 2017. Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics, 33(14):i37–i48. find
  28. Kristina M Hettne, Rob H Stierum, Martijn J Schuemie, Peter JM Hendriksen, Bob JA Schijvenaars, Erik M van Mulligen, Jos Kleinjans, and Jan A Kors. 2009. A dictionary to identify small molecules and drugs in free text. Bioinformatics, 25(22):2983–2991. find
  29. Lynette Hirschman, Alexander Yeh, Christian Blaschke, and Alfonso Valencia. 2005. Overview of biocreative: critical assessment of information extraction for biology. find
  30. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991. find our card
  31. Jin-Dong Kim, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Nigel Collier. 2004. Introduction to the bio-entity recognition task at jnlpba. In Proceedings of the international joint workshop on natural language processing in biomedicine and its applications, pages 70–75. Association for Computa… find our card
  32. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2016. Character-aware neural language models. In AAAI, pages 2741–2749. find our card
  33. Craig Knox, Vivian Law, Timothy Jewison, Philip Liu, Son Ly, Alex Frolkis, Allison Pon, Kelly Banco, Christine Mak, Vanessa Neveu, et al. 2010. Drugbank 3.0: a comprehensive resource for omics research on drugs. Nucleic acids research, 39(suppl 1):D1035–D1041. find
  34. Martin Krallinger, Obdulia Rabal, Florian Leitner, Miguel Vazquez, David Salgado, Zhiyong Lu, Robert Leaman, Yanan Lu, Donghong Ji, Daniel M Lowe, et al. 2015. The chemdner corpus of chemicals and drugs and its annotation principles. Journal of cheminformatics, 7(S1):S2. find
  35. Onur Kuru, Ozan Arkan Can, and Deniz Yuret. 2016. Charner: Character-level named entity recognition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 911–921. find
  36. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360. find
  37. Robert Leaman and Graciela Gonzalez. 2008. Banner: an executable survey of advances in biomedical named entity recognition. In Biocomputing 2008, pages 652–663. World Scientific. find
  38. Jianbo Lei, Buzhou Tang, Xueqin Lu, Kaihua Gao, Min Jiang, and Hua Xu. 2013. A comprehensive study of named entity recognition in chinese clinical text. Journal of the American Medical Informatics Association, 21(5):808–814. find
  39. Yaoyong Li, Kalina Bontcheva, and Hamish Cunningham. 2005. Svm based learning system for information extraction. In Deterministic and statistical methods in machine learning, pages 319–339. Springer. find
  40. Yanran Li, Wenjie Li, Fei Sun, and Sujian Li. 2015. Component-enhanced chinese character embeddings. arXiv preprint arXiv:1508.06669. find
  41. Nut Limsopatham and Nigel Henry Collier. 2016. Bidirectional lstm for named entity recognition in twitter messages. find
  42. Wang Ling, Tiago Luı́s, Luı́s Marujo, Ramón Fernandez Astudillo, Silvio Amir, Chris Dyer, Alan W Black, and Isabel Trancoso. 2015. Finding function in form: Compositional character models for open vocabulary word representation. arXiv preprint arXiv:1508.02096. find our card
  43. Shengyu Liu, Buzhou Tang, Qingcai Chen, and Xiaolong Wang. 2015. Effects of semantic features on machine learning-based drug name recognition systems: word embeddings vs. manually constructed dictionaries. Information, 6(4):848–865. 2015. Joint Named Entity Recognition and Disambiguation, September. find
  44. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354. find our card
  45. Robert Malouf. 2002. Markov models for language-independent named entity recognition, proceedings of the 6th conference on natural language learning. August, 31:1–4. find
  46. Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Computational linguistics, 19(2):313–330. find our card
  47. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. find
  48. Shotaro Misawa, Motoki Taniguchi, Yasuhide Miura, and Tomoko Ohkuma. 2017. Character-based bidirectional lstm-crf with words and characters for japanese named entity recognition. In Proceedings of the First Workshop on Subword and Character Level Models in NLP, pages 97–102. find
  49. David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes, 30(1):3–26. find
  50. David Nadeau, Peter D Turney, and Stan Matwin. 2006. Unsupervised named-entity recognition: Generating gazetteers and resolving ambiguity. In Conference of the Canadian Society for Computational Studies of Intelligence, pages 266–277. Springer. find
  51. TS Nguyen, LM Nguyen, and XC Tran. 2016. Vietnamese named entity recognition at vlsp 2016 evaluation campaign. In Proceedings of The Fourth International Workshop on Vietnamese Language and Speech Processing. find
  52. Tomoko Ohta, Yuka Tateisi, and Jin-Dong Kim. 2002. The genia corpus: An annotated research abstract corpus in molecular biology domain. In Proceedings of the second international conference on Human Language Technology Research, pages 82–86. Morgan Kaufmann Publishers Inc. find
  53. Alexandre Passos, Vineet Kumar, and Andrew McCallum. 2014. Lexicon infused phrase embeddings for named entity resolution. arXiv preprint arXiv:1404.5367. find our card
  54. Nita Patil, Ajay S Patil, and BV Pawar. 2016. Survey of named entity recognition systems with respect to indian and foreign languages. International Journal of Computer Applications, 134(16). find
  55. Thai-Hoang Pham and Phuong Le-Hong. 2017. End-to-end recurrent neural network models for vietnamese named entity recognition: Word-level vs. character-level. arXiv preprint arXiv:1705.04044. — [arXiv:1705.04044](https://arxiv.org/abs/1705.04044) find
  56. Jakub Piskorski, Lidia Pivovarova, Jan Šnajder, Josef Steinberger, Roman Yangarber, et al. 2017. The first cross-lingual challenge on recognition, normalization and matching of named entities in slavic languages. In Proceedings of the 6th Workshop on Balto-Slavic Natural Language Processing. Associ… find
  57. Barbara Plank, Anders Søgaard, and Yoav Goldberg. 2016. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. arXiv preprint arXiv:1604.05529. find our card
  58. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using ontonotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 143–152. find
  59. Lawrence Rabiner and B Juang. 1986. An introduction to hidden markov models. ieee assp magazine, 3(1):4–16. find
  60. Dipti Misra Sharma Rajeev Sangal and Anil Kumar Singh, editors. 2008. Proceedings of the IJCNLP-08 Workshop on Named Entity Recognition for South and South East Asian Languages. Asian Federation of Natural Language Processing, Hyderabad, India, January. find
  61. Tim Rocktäschel, Torsten Huber, Michael Weidlich, and Ulf Leser. 2013. Wbi-ner: The impact of domain-specific features on the performance of identifying and classifying mentions of drugs. In SemEval@ NAACL-HLT, pages 356–363. find
  62. Sujan Kumar Saha, Sudeshna Sarkar, and Pabitra Mitra. 2009. Feature selection techniques for maximum entropy based biomedical named entity recognition. Journal of Biomedical Informatics, 42(5):905 – 911. Biomedical Natural Language Processing. find
  63. Diana Santos and Nuno Cardoso. 2007. Reconhecimento de entidades mencionadas em português: Documentação e actas do harem, a primeira avaliação conjunta na área. find
  64. Cicero Nogueira dos Santos and Victor Guimaraes. 2015. Boosting named entity recognition with neural character embeddings. arXiv preprint arXiv:1505.05008. find
  65. Robert E Schapire. 2013. Explaining adaboost. In Empirical inference, pages 37–52. Springer. find
  66. Isabel Segura Bedmar, Paloma Martı́nez, and Marı́a Herrero Zazo. 2013. Semeval-2013 task 9: Extraction of drug-drug interactions from biomedical texts (ddiextraction 2013). Association for Computational Linguistics. find
  67. Khaled Shaalan. 2014. A survey of arabic named entity recognition and classification. Computational Linguistics, 40(2):469–510. find
  68. Rahul Sharnagat. 2014. Named entity recognition: A literature survey. Center For Indian Language Technology. find
  69. Stephanie Strassel, Alexis Mitchell, and Shudong Huang. 2003. Multilingual resources for entity extraction. In Proceedings of the ACL 2003 workshop on Multilingual and mixed-language named entity recognition-Volume 15, pages 49–56. Association for Computational Linguistics. find
  70. Koichi Takeuchi and Nigel Collier. 2002. Use of support vector machines in extended named entity recognition. In proceedings of the 6th conference on Natural language learning-Volume 20, pages 1–7. Association for Computational Linguistics. find
  71. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 142–147. Association for Computational Linguistics. find
  72. Erik F Tjong Kim Sang. 2002. Introduction to the conll-2002 shared task: language-independent named entity recognition, proceedings of the 6th conference on natural language learning. August, 31:1–4. find our card
  73. Özlem Uzuner, Yuan Luo, and Peter Szolovits. 2007. Evaluating the state-of-the-art in automatic de-identification. Journal of the American Medical Informatics Association, 14(5):550–563. find
  74. Özlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2011. 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552– 556. find
  75. Kai Xu, Zhanfan Zhou, Tianyong Hao, and Wenyin Liu. 2017. A bidirectional lstm and conditional random fields approach to medical named entity recognition. In International Conference on Advanced Intelligent Systems and Informatics, pages 355–365. Springer. find
  76. Vikas Yadav, Rebecca Sharp, and Steven Bethard. 2018. Deep affix features improve neural named entity recognizers. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 167–172. find
  77. Shao Yan, Christian Hardmeier, and Joakim Nivre. 2016. Multilingual named entity recognition using hybrid neural networks. In The Sixth Swedish Language Technology Conference (SLTC). find
  78. Zhilin Yang, Ruslan Salakhutdinov, and William Cohen. 2016. Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv:1603.06270. — [arXiv:1603.06270](https://arxiv.org/abs/1603.06270) find
  79. Zhilin Yang, Ruslan Salakhutdinov, and William W Cohen. 2017. Transfer learning for sequence tagging with hierarchical recurrent networks. arXiv preprint arXiv:1703.06345. — [arXiv:1703.06345](https://arxiv.org/abs/1703.06345) find
  80. Rongchao Yin, Quan Wang, Peng Li, Rui Li, and Bin Wang. 2016. Multi-granularity chinese word embedding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 981–986. find
  81. Shaodian Zhang and Noémie Elhadad. 2013. Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts. Journal of biomedical informatics, 46(6):1088–1098. find
  82. GuoDong Zhou and Jian Su. 2002. Named entity recognition using an hmm-based chunk tagger. In proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 473–480. Association for Computational Linguistics. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yadav-2018-survey-recent-advances-named-entity/