Elamur Research⌕
☰
home / papers / 2023 / LIPN at WojoodNER shared task: A Span-Based Approach for Flat and Nested Arabic Named Entity Recognition

LIPN at WojoodNER shared task: A Span-Based Approach for Flat and Nested Arabic Named Entity Recognition

Paper · 2023 · Niama Elkhbir, Urchade Zaratiana, Nadi Tomeh et al.

Summary

The paper addresses Arabic named entity recognition in both its flat and nested forms. Nested Arabic NER has seen little work because suitable annotated data was scarce. The authors entered the WojoodNER 2023 shared task, which uses the Wojood corpus: about 550k tokens covering MSA and dialects, annotated with 21 entity types in flat and nested versions. They cast NER as classifying every candidate span up to 10 tokens long.

  • Encoding: tokens are encoded with bert-base-arabert followed by a bi-LSTM, and each span is represented by a 1D convolution over its tokens.
  • Scoring: a linear layer per label scores each span representation.
  • Flat decoding: spans predicted as non-entities are discarded, then a maximum independent set algorithm picks the best non-overlapping set.
  • Nested decoding: a greedy procedure repeatedly takes the highest-scoring span that does not cross spans already chosen.

The system ranked first in flat NER (F1 91.96) and fourth in nested NER (F1 92.45) out of eleven teams. The work shows that a single span-scoring model can serve both structural regimes simply by swapping the decoder.

Key points

  • Wojood train split: 16,817 sentences, 394,500 tokens, 50,032 flat and 62,403 nested entities. The flat labels come from the outermost label in the nested annotation.
  • Training setup: learning rate 5e-6 for AraBERT and 1e-3 for the other layers, Adam, 50,000 steps, batch size 12, roughly 50 GPU hours on V100s.
  • Capping spans at 10 tokens drops 140 training and 19 validation entities. This limits maximum recall to 99.72 (train) and 99.73 (validation).
  • Weakest flat classes were PRODUCT (60.00) and WEBSITE (63.77). Several classes are tiny; for example, CURR makes up 0.04% and UNIT 0.01% of the training data.
  • Error analysis on the flat validation set found three kinds of error:
    • 68 spans with correct boundaries but the wrong label, about 45% of them LOC/ORG/GPE confusions over place names;
    • 167 boundary errors;
    • 305 predicted spans missing from the gold data, many of which look like genuine entities the annotators missed.

Interesting ideas

  • I1. One span scorer feeds different structure-enforcing decoders: an exact non-overlap search for flat output, greedy non-crossing selection for nested output; the model stays the same and only the constraint changes.
  • I2. Removing spans labelled "no entity" before running the maximum independent set keeps the exact search small and cheap.
  • I3. Greedy picking of the best-scoring span that does not cross existing choices yields properly nested structures, such as an organization containing a city.
  • I4. The authors compute the recall ceiling that a design limit (the span-length cap) imposes, so lost recall is attributed to the design rather than the model.
  • I5. Errors are sorted into right-span/wrong-label, right-label/wrong-boundary, and extra predictions that may expose gaps in the gold data.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): role-argument candidates for a predicate could be scored once, then resolved by a non-overlap decoder, so each chosen argument is traceable to its score and the constraint applied.
  • I2 (world): when resolving competing entity mentions, prune clearly negative candidates first, then run an exact conflict-free selection over the survivors.
  • I3 (world): build nested entity structures (e.g., "University of Kyiv" containing "Kyiv") as layered nodes in the text world instead of forcing one flat label.
  • I4 (learn): every evaluation report could state the ceilings set by hard limits (max span, vocabulary, parser coverage) next to the scores.
  • I5 (learn): apply this taxonomy when witness parsers disagree on silver data, so missing gold annotations get flagged for repair instead of being counted as model mistakes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🔬Laboratoire d'Informatique de Paris-NordVilletaneuse · France9 works · 510 citations🏢FI GroupLisbon · Portugal2 works · 196 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksNamed entity recognition · 200 works

Cite

Cite as Niama Elkhbir, Urchade Zaratiana, Nadi Tomeh and Thierry Charnois. 2023. LIPN at WojoodNER shared task: A Span-Based Approach for Flat and Nested Arabic Named Entity Recognition. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.89

@inproceedings{elkhbir2023lipn,
  title = {{LIPN at WojoodNER shared task: A Span-Based Approach for Flat and Nested Arabic Named Entity Recognition}},
  author = {Niama Elkhbir, Urchade Zaratiana, Nadi Tomeh and Thierry Charnois},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.89},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.89}
}

References

33 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Manel Affi and Chiraz Latiri. 2022. Arabic named entity recognition using variant deep neural network architectures and combinatorial feature embedding based on cnn, lstm and bert. In Proceedings of the 36th Pacific Asia Conference on Language, Information and Computation, pages 302–312.
  2. Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018. Contextual string embeddings for sequence labeling. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1638– 1649, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  3. Muhammad Saleh Al-Qurishi and Riad Souissi. 2021. Arabic named entity recognition using transformer-based-crf model. In Proceedings of the 4th International Conference on Natural Language and Speech Processing (ICNLSP 2021), pages 262–271.
  4. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  5. Mikhail Arkhipov, Maria Trofimova, Yuri Kuratov, and Alexey Sorokin. 2019. Tuning multilingual transformers for language-specific named entity recognition. In Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing, pages 89–93, Florence, Italy. Association for Computational Linguistics.
  6. Sławomir Dadas and Jarosław Protasiewicz. 2020. A bidirectional iterative algorithm for nested named entity recognition. IEEE Access, 8:135091–135102. ↗
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  8. Niama El Khbir, Nadi Tomeh, and Thierry Charnois. 2022. Arabie: Joint entity, relation and event extraction for arabic. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 331–345.
  9. Jinlan Fu, Xuanjing Huang, and Pengfei Liu. 2021. SpanNER: Named entity re-/recognition as span prediction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7183–7195, Online. Association for Computational Linguistics. ↗
  10. Chadi Helwe, Ghassan Dib, Mohsen Shamas, and Shady Elbassuoni. 2020. A semi-supervised BERT approach for Arabic named entity recognition. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 49–57, Barcelona, Spain (Online). Association for Computational Linguistics.
  11. Mustafa Jarrar, Muhammad Abdul-Mageed, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, Nagham Hamad, and Alaa Omer. 2023. WojoodNER: The Arabic Named Entity Recognition Shared Task. In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023). find
  12. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested Arabic named entity corpus and recognition using BERT. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3626–3636, Marseille, France. European Language Resources Association.
  13. Arzoo Katiyar and Claire Cardie. 2018. Nested named entity recognition revisited. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 861–871, New Orleans, Louisiana. Association for Computational Linguistics.
  14. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260–270, San Diego, California. Association for Computational Linguistics. ↗
  15. Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020. A unified mrc framework for named entity recognition.
  16. Liyuan Liu, Jingbo Shang, and Jiawei Han. 2019. Arabic named entity recognition: What works and what’s next. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 60–67, Florence, Italy. Association for Computational Linguistics.
  17. Yijia Liu, Wanxiang Che, Jiang Guo, Bing Qin, and Ting Liu. 2016. Exploring segment representations for neural segmentation models. In IJCAI. ↗
  18. Cedric Lothritz, Kevin Allix, Lisa Veiber, Tegawendé F. Bissyandé, and Jacques Klein. 2020. Evaluating pre-trained transformer-based models on the task of fine-grained named entity recognition. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3750–3760, Barcelona, Spain (Online). International Committee on Computational Linguistics. find
  19. Andrew McCallum and Wei Li. 2003. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons.
  20. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  21. Dan Shen, Jie Zhang, Guodong Zhou, Jian Su, and Chew-Lim Tan. 2003. Effective adaptation of hidden Markov model-based named entity recognizer for biomedical domain. In Proceedings of the ACL 2003 Workshop on Natural Language Processing in Biomedicine, pages 49–56, Sapporo, Japan. Association for Computational Linguistics. find
  22. Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2022b. GNNer: Reducing overlapping in span-based NER using graph neural networks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 97–103, Dublin, Ireland. Association for Computational Linguistics. find
  23. Mohammad Golam Sohrab and Makoto Miwa. 2018. Deep exhaustive model for nested named entity recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2843–2849, Brussels, Belgium. Association for Computational Linguistics.
  24. Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2022c. Named entity recognition as structured span prediction. In Proceedings of the Workshop on Unimodal and Multimodal Induction of Linguistic Structures (UM-IoS), pages 1–10, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find
  25. Koichi Takeuchi and Nigel Collier. 2002. Use of support vector machines in extended named entity recognition. In COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002). find
  26. Jue Wang, Lidan Shou, Ke Chen, and Gang Chen. 2020. Pyramid: A layered model for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5918–5928, Online. Association for Computational Linguistics. find
  27. Congying Xia, Chenwei Zhang, Tao Yang, Yaliang Li, Nan Du, Xian Wu, Wei Fan, Fenglong Ma, and Philip Yu. 2019. Multi-grained named entity recognition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1430– 1440, Florence, Italy. Association for Computational Linguistics. find
  28. Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, and Xipeng Qiu. 2021. A unified generative framework for various ner subtasks. In ACL. ↗
  29. Songlin Yang and Kewei Tu. 2022. Bottom-up constituency parsing and nested named entity recognition with pointer networks. In ACL. find
  30. Zhiwei Yang, Jing Ma, Hechang Chen, Yunke Zhang, and Yi Chang. 2021. HiTRANS: A hierarchical transformer network for nested named entity recognition. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 124–132, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  31. Jie Zhang, Dan Shen, Guodong Zhou, Jian Su, and Chew-Lim Tan. 2004. Enhancing hmm-based biomedical named entity recognition by studying special phenomena. Journal of Biomedical Informatics, 37(6):411–422. Named Entity Recognition in Biomedicine. ↗
  32. Changmeng Zheng, Yi Cai, Jingyun Xu, Ho-fung Leung, and Guandong Xu. 2019. A boundary-aware neural model for nested named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 357–366, Hong Kong, China. Association for Computational Linguistics. find
  33. GuoDong Zhou and Jian Su. 2002. Named entity recognition using an hmm-based chunk tagger. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, page 473–480, USA. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elkhbir-2023-lipn-wojoodner-shared-task-span/