Elamur Research⌕
☰
home / papers / 2026 / Learning Nested Named Entity Recognition from Flat Annotations

Learning Nested Named Entity Recognition from Flat Annotations

Paper · 2026 · Igor Rozhkov and Natalia V Loukachevitch

Summary

Nested NER, where entities sit inside other entities, depends on costly multi-level annotation, so nested corpora are rare while flat NER datasets are plentiful. The authors ask whether nested structure can be learned from flat labels alone. They use NEREL, a Russian news benchmark with 29 entity types in which about a fifth of mentions are nested, reduced to outermost mentions only. They test four approaches, mostly with a Binder span-based model. The first adds inner mentions found by substring matching against other annotated entities, optionally after pymorphy2 lemmatization. The second corrupts words in long entities to elicit pseudo-nested predictions through 5-fold retraining. The third excludes plausible inner spans from the negative loss. The fourth is a hybrid in which Binder finds outer entities and an LLM labels inner ones. Combining the three fine-tuning techniques gives the best inner-entity F1. According to the authors, this recovers roughly 40% of the performance difference between the flat baseline and a model trained with complete nested annotations. LLM-based setups lag behind. The work shows that existing flat resources can partly substitute for nested annotation.

Key points

  • Inclusions: substring matching alone lifts inner-entity F1 from 3.84% (flat baseline) to 21.36%.
  • Combined method: inclusions, entity corruption and flat neutralization together reach 26.37% inner F1. Full nested supervision reaches 65.48%.
  • Entity corruption: five symbol types were tested (digits, letters, mixed, semicolons, commas). Corrupting the end position of an entity consistently worked best.
  • LLM approaches: DeepSeek-R1 and RuAdapt-Qwen2.5 were used.
    • The hybrid Binder + LLM pipeline gets 70.16% overall F1 but only 18.84% inner F1.
    • A pure few-shot LLM gets 42.39% overall and 5.78% inner F1, compared with 76.59% overall for fine-tuned models.
  • Limitations and code: the study covers only Russian news (NEREL), one architecture (Binder) and three runs per configuration. Code is available on GitHub (fulstock/Learning-from-Flat-Annotations).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Lomonosov Moscow State UniversityMoscow · Russia14 works · 2,430 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Named entity recognition · 200 works

Cite

Cite as Igor Rozhkov and Natalia V Loukachevitch. 2026. Learning Nested Named Entity Recognition from Flat Annotations. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-srw.50

@inproceedings{rozhkov2026learning,
  title = {{Learning Nested Named Entity Recognition from Flat Annotations}},
  author = {Igor Rozhkov and Natalia V Loukachevitch},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-srw.50},
  url = {https://doi.org/10.18653/v1/2026.eacl-srw.50}
}

References

36 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ekaterina Artemova, Natalia Loukachevitch, and Pavel Braslavski. 2022. RuNNE-2022 shared task: Recognizing nested named entities. In Computational Linguistics and Intellectual Technologies: Papers from the Annual Conference “Dialogue”. find
  2. Xiang Dai and Heike Adel. 2020. An analysis of simple data augmentation for named entity recognition. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3861–3867, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  3. Vera Davydova, Natalia Loukachevitch, and Elena Tutubalina. 2024. Overview of BioNNE task on biomedical nested named entity recognition at BioASQ 2024. In CLEF Working Notes. find
  4. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The automatic content extraction (ACE) program – tasks, data, and evaluation. In Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04), Lisbo…
  5. Jenny Rose Finkel and Christopher D. Manning. 2009. Nested named entity recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 141–150, Singapore. Association for Computational Linguistics.
  6. Daya Guo, Dejian Yang, He Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948.
  7. Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu, and Xiang Wan. 2023. Is information extraction solved by ChatGPT? an analysis of performance, evaluation criteria, robustness and errors. arXiv preprint arXiv:2305.14450. — [arXiv:2305.14450](https://arxiv.org/abs/2305.14450) find
  8. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested Arabic named entity corpus and recognition using BERT. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3626–3636, Marseille, France. European Language Resources Association.
  9. Mikhail Korobov. 2015. Morphological analyzer and generator for Russian and Ukrainian languages. In Analysis of Images, Social Networks and Texts, pages 320–332. Springer. find
  10. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, page…
  11. Pierre Lison, Jeremy Barnes, Aliaksandr Hubin, and Samia Touileb. 2020. Named entity recognition without labelled data: A weak supervision approach. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1518–1533, Online. Association for Computational Ling… find
  12. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR). find
  13. Natalia Loukachevitch, Ekaterina Artemova, Tatiana Batura, Pavel Braslavski, Ilia Denisov, Vladimir Ivanov, Suresh Manandhar, Alexander Pugachev, and Elena Tutubalina. 2021. NEREL: A Russian dataset with nested named entities, relations and events. In Proceedings of the International Conference on R… find
  14. Wei Lu and Dan Roth. 2015. Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 857–867, Lisbon, Portugal. Association for Computational Linguistics.
  15. Meizhi Ju, Makoto Miwa, and Sophia Ananiadou. 2018. A neural layered model for nested named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1446–1459,…
  16. Yu Meng, Yunyi Zhang, Jiaxin Huang, Xuan Wang, Yu Zhang, Heng Ji, and Jiawei Han. 2021. Distantly-supervised named entity recognition with noise-robust learning and language model augmented self-training. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, page… find
  17. Hongjin Kim, Jai-Eun Kim, and Harksoo Kim. 2024. Exploring nested named entity recognition with large language models: Methods, challenges, and insights. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8653–8670, Miami, Florida, USA. Association for C… find
  18. Barbara Plank, Kristian Nørgaard Jensen, and Rob van der Goot. 2020. DaN+: Danish nested named entities and lexical normalization. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6649–6662, Barcelona, Spain (Online). International Committee on Computational Li… find
  19. Jin-Dong Kim, Tomoko Ohta, Yuka Tateisi, and Jun’ichi Tsujii. 2003. GENIA corpus—a semantically annotated corpus for bio-textmining. Bioinformatics, 19(Supplement 1):i180–i182. find
  20. Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), pages 147–155, Boulder, Colorado. Association for Computational Linguistics.
  21. Nicky Ringland, Xiang Dai, Ben Hachey, Sarvnaz Karimi, Cecile Paris, and James R. Curran. 2019. NNE: A dataset for nested named entity recognition in English newswire. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5176–5181, Florence, Italy. Associ…
  22. Igor Rozhkov and Natalia Loukachevitch. 2023. Prompts in few-shot named entity recognition. Pattern Recognition and Image Analysis, 33(2):238–244. find
  23. Igor Rozhkov and Natalia Loukachevitch. 2025. Methods for recognizing nested terms. In Proceedings of the 2025 Conference. find
  24. Zeqi Tan, Yongliang Shen, Shuai Zhang, Weiming Lu, and Yueting Zhuang. 2021. A sequence-to-set network for nested named entity recognition. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21), pages 3936–3942. find
  25. Mikhail Tikhomirov and Daniil Chernyshev. 2023. Impact of tokenization on LLaMA Russian adaptation. In 2023 Ivannikov Ispras Open Conference (ISPRAS), pages 163–168. IEEE. find
  26. Mikhail Tikhomirov and Daniil Chernyshov. 2024. Facilitating large language model Russian adaptation with learned embedding propagation. Journal of Language and Education, 10(4):130–145. find
  27. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142–147.
  28. Solenn Tual and 1 others. 2023. A benchmark of nested named entity recognition approaches in historical structured documents. In International Conference on Document Analysis and Recognition, pages 115–131. find
  29. Jue Wang, Lidan Shou, Ke Chen, and Gang Chen. 2020. Pyramid: A layered model for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5918–5928, Online. Association for Computational Linguistics. find
  30. Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, and Guoyin Wang. 2023. GPT-NER: Named entity recognition via large language models. arXiv preprint arXiv:2304.10428.
  31. Yukun Yan, Tao Gui, Junqi Ye, and Qi Zhang. 2023. Nested named entity recognition as building local hypergraphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13855–13863. find
  32. Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6470–6476, Online. Association for Computational Linguistics.
  33. Hao Zhang and 1 others. 2023a. Judicial nested named entity recognition method with MRC framework. International Journal of Cognitive Computing in Engineering, 4:118–126. find
  34. Sheng Zhang, Hao Cheng, Jianfeng Gao, and Hoifung Poon. 2023b. Optimizing bi-encoder for named entity recognition via contrastive learning. In The Eleventh International Conference on Learning Representations (ICLR). find
  35. Enwei Zhu, Yiyang Sheng, Yanping Chen, and Jinpeng Li. 2022. Recognizing nested entities from flat supervision: A new NER subtask, feasibility and challenges. arXiv preprint arXiv:2211.11116. — [arXiv:2211.11116](https://arxiv.org/abs/2211.11116) find
  36. Dmitry Zmitrovich, Aleksandr Abramov, Andrey Kalmykov, Vitaly Kadulin, Maria Tikhonova, Ekaterina Taktasheva, Danil Astafurov, Mark Baushenko, Artem Snegirev, Tatiana Shavrina, Sergei S. Markov, Vladislav Mikhailov, and Alena Fenogenova. 2024. A family of pretrained transformer language models for R…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rozhkov-2026-learning-nested-named-entity-recognition/