Elamur Research⌕
☰
home / papers / 2023 / Arabic Fine-Grained Entity Recognition

Arabic Fine-Grained Entity Recognition

Paper · 2023 · Haneen Abdallatif Liqreina, Mustafa Jarrar, Mohammed Khalilia et al.

Summary

Most named-entity recognizers assign only broad labels such as organization or location. This paper adds a finer layer for Arabic. Previously, the only Arabic resource with subtypes was the costly and ageing ACE2005. The authors instead extended the open Wojood corpus (550K tokens, 21 nested entity types, mostly MSA plus Palestinian and Lebanese dialect text from social media).

The work had two steps:

  • They re-annotated the GPE, LOC, ORG and FAC mentions to follow the LDC's ACE guidelines, which required 5,614 corrections.
  • They manually assigned all of those mentions (about 44K) to 31 subtypes, mostly from ACE plus a few new tags.

The result, Wojood_Fine, shows very high annotator agreement. For baselines, they fine-tuned three Arabic BERT encoders on three tasks: recognizing non-overlapping entities, recognizing overlapping (nested) entities, and a multi-task setup that predicts nested entities together with their subtypes. ARABERTv2 performed best on the two nested tasks. The corpus and models are openly released. This makes Wojood_Fine the first freely available fine-grained, nested NER resource covering both MSA and dialectal Arabic.

Key points

  • Wojood_Fine contains roughly 44.4K GPE/ORG/LOC/FAC mentions. About 47.6K tokens carry one of 31 subtypes based on the ACE 2008 Arabic guidelines (V7.4.2). The authors added NEIGHBORHOOD, CAMP, SPORT and ORG_FAC to the ACE set.
  • Inter-annotator agreement reached Cohen's Kappa 0.9861 and F1 0.9889.
  • The encoders tested were ARBERTv2, MARBERTv2 and ARABERTv2. The best F1 scores were 0.920 for flat NER, 0.866 for nested NER and 0.885 for nested NER with subtypes.
  • Mentions of a country that refer to its government were moved from ORG to GPE with the subtype GPE_ORG.
  • The largest error class on Dev was "wrong tag", followed by "no prediction". Within wrong tags, about 14.3% came from dialectal words, with a similar share from nesting. Performance also drops on out-of-domain text such as finance and science.

Interesting ideas

  • I1. Metonymic readings stay under a stable top-level type and are separated only at the subtype level (Country vs GPE_ORG for "Nigeria"), so the coarse label never flips with context.
  • I2. Before adding a finer layer, the authors first realigned the old annotations to an established external guideline, logging thousands of corrections, so the extension stays compatible with other corpora.
  • I3. The error taxonomy separates "right parent, wrong/missing subtype" from "right subtype, missing parent", alongside dialect confusion and ordinal/cardinal mix-ups. This shows exactly which level of a hierarchy fails.
  • I4. Agreement was reported both as Kappa and as span-level F1, combining a chance-corrected statistic with one that is comparable to model scores.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): an entity in the world of a text could keep one identity while a sense facet records whether it acts as a place or as a governing agent. Frames can then treat "Nigeria decided" as an agent without creating a new entity.
  • I2 (tree): when merging treebanks or converting UD dialects, normalise to one reference convention first and keep a diff count as an explicit, auditable conversion cost.
  • I3 (learn): hierarchical outputs such as UD relation subtypes (nmod:poss) or VerbNet class/subclass could be scored with the same split, so repair rules target the failing level.
  • I4 (learn): measuring agreement among witness parsers in both ways would show whether silver-data consensus is genuine or partly due to label skew.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of British ColumbiaVancouver · Canada98 works · 34,805 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations🎓Birzeit UniversityBirzeit · Palestine36 works · 798 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksNamed entity recognition · 200 works

Cite

Cite as Haneen Abdallatif Liqreina, Mustafa Jarrar, Mohammed Khalilia, Ahmed Oumar El-Shangiti and Muhammad Abdul-Mageed. 2023. Arabic Fine-Grained Entity Recognition. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.25

@inproceedings{liqreina2023arabic,
  title = {{Arabic Fine-Grained Entity Recognition}},
  author = {Haneen Abdallatif Liqreina, Mustafa Jarrar, Mohammed Khalilia, Ahmed Oumar El-Shangiti and Muhammad Abdul-Mageed},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.25},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.25}
}

References

38 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  2. Barbara Di Eugenio and Michael Glass. 2004. The Kappa Statistic: A Second Look. Computational Linguistics, 30(1):95–101. ↗
  3. Besnik Fetahu, Sudipta Kar, Zhiyu Chen, Oleg Rokhlenko, and Shervin Malmasi. 2023. Semeval-2023 task 2: Fine-grained multilingual named entity recognition (multiconer 2). arXiv preprint arXiv:2305.06586. ↗
  4. Moustafa Al-Hajj and Mustafa Jarrar. 2021. Arabgloss-bert: Fine-tuning bert on context-gloss pairs for wsd. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 40–48, Online. INCOMA Ltd. find
  5. Sana Ghanem, Mustafa Jarrar, Radi Jarrar, and Ibrahim Bounhas. 2023. A benchmark and scoring algorithm for enriching arabic synonyms. In Proceedings of the 12th International Global Wordnet Conference (GWC2023), pages 215–222. Global Wordnet Association.
  6. Diana Alhafi, Anton Deik, and Mustafa Jarrar. 2019. Usability evaluation of lexicographic e-services. In The 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), pages 1–7. IEE. find
  7. Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras + baladi: Towards a levantine corpus. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France. ↗
  8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. Arabert: Transformer-based model for arabic language understanding. ↗
  9. Ahmed Hamdi, Elvys Linhares Pontes, Emanuela Boros, Thi Tuyet Hai Nguyen, Günter Hackl, Jose G Moreno, and Antoine Doucet. 2021. A multilingual dataset for named entity recognition, entity linking and stance detection in historical newspapers. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2328–2334. find
  10. Mamoun Abu Helou, Matteo Palmonari, and Mustafa Jarrar. 2016. Effectiveness of automatic translations for cross-lingual ontology mapping. Journal of Artificial Intelligence Research, 55(1):165–208. ↗
  11. Alankar Jain, Bhargavi Paranjape, and Zachary C Lipton. 2019. Entity projection via machine translation for cross-lingual ner. arXiv preprint arXiv:1909.05356. ↗
  12. Mustafa Jarrar. 2011. Building a formal arabic ontology (invited paper). In Proceedings of the Experts Meeting on Arabic Ontologies and Semantic Networks. ALECSO, Arab League. find
  13. Mustafa Jarrar. 2021. The arabic ontology - an arabic wordnet with ontologically clean content. Applied Ontology Journal, 16(1):1–26. ↗
  14. Mustafa Jarrar and Hamzeh Amayreh. 2019. An arabic-multilingual database with a lexicographic search engine. In The 24th International Conference on Applications of Natural Language to Information Systems (NLDB 2019), volume 11608 of LNCS, pages 234–246. Springer. find
  15. Mustafa Jarrar, Hamzeh Amayreh, and John P. McCrae. 2019. Representing arabic lexicons in lemon - a preliminary study. In The 2nd Conference on Language, Data and Knowledge (LDK 2019), volume 2402, pages 29–33. CEUR Workshop Proceedings. ↗
  16. Mustafa Jarrar, Anton Deik, and Bilal Faraj. 2011. Ontology-based data and process governance framework -the case of e-government interoperability in palestine. In Proceedings of the IFIP International Symposium on Data-Driven Process Discovery and Analysis (SIMPDA’11), pages 83–98. find
  17. Mustafa Jarrar, Nizar Habash, Diyam Akra, and Nasser Zalmout. 2014. Building a corpus for palestinian arabic: a preliminary study. In Proceedings of the EMNLP 2014, Workshop on Arabic Natural Language, pages 18–27. Association For Computational Linguistics.
  18. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: An annotated corpus for the palestinian arabic dialect. Journal Language Resources and Evaluation, 51(3):745– 775. ↗
  19. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested arabic named entity corpus and recognition using bert. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
  20. Mustafa Jarrar, Sanad Malaysha, Tymaa Hammouda, and Mohammad Khalilia. 2023a. Salma: Arabic sense-annotated corpus and wsd benchmarks. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023. ACL.
  21. Mustafa Jarrar, Fadi Zaraket, Tymaa Hammouda, Daanish Masood Alavi, and Martin Waahlisch. 2023b. Lisan: Yemeni, irqi, libyan, and sudanese arabic dialect copora with morphological annotations. In The 20th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA). IEEE. find
  22. Ridong Jiang, Rafael E. Banchs, and Haizhou Li. 2016. Evaluating and combining name entity recognition systems. In Proceedings of the Sixth Named Entity Workshop, pages 21–27, Berlin, Germany. Association for Computational Linguistics. find
  23. Diksha Khurana, Aditya Koli, Kiran Khatter, and Sukhdev Singh. 2022. Natural language processing: State of the art, current trends and challenges. Multimedia Tools and Applications, 82. ↗
  24. Changki Lee, Yi-Gyu Hwang, Hyo-Jung Oh, Soojong Lim, Jeong Heo, Chung-Hee Lee, Hyeon-Jin Kim, Ji-Hyun Wang, and Myung-Gil Jang. 2006. Fine-grained named entity recognition using conditional random fields for question answering. In Information Retrieval Technology, pages 581–587, Berlin, Heidelberg. Springer Berlin Heidelberg. find
  25. Jing li, Aixin Sun, Ray Han, and Chenliang Li. 2020. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, PP:1–1. find
  26. Xiao Ling and Daniel Weld. 2012. Fine-grained entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, pages 94–100. find
  27. Bang Liu, Haojie Wei, Di Niu, Haolan Chen, and Yancheng He. 2020. Asking questions the human way: Scalable question-answer generation from text corpus. In Proceedings of The Web Conference 2020, pages 2032–2043. find
  28. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. find
  29. Amal Nayouf, Mustafa Jarrar, Fadi zaraket, Tymaa Hammouda, and Mohamad-Bassam Kurdy. 2023. Nâbra: Syrian arabic dialects with morphological annotations. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023. ACL.
  30. Ramzi Esmail Salah and Lailatul Qadri Binti Zakaria. 2018. Building the classical arabic named entity recognition corpus (canercorpus). Journal of Theoretical and Applied Information Technology, 96. find
  31. Erik F Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050.
  32. Erik F. Tjong Kim Sang and Jorn Veenstra. 1999. Representing text chunks. In Proceedings of the Ninth Conference on European Chapter of the Association for Computational Linguistics, EACL ’99, page 173–179, USA. Association for Computational Linguistics.
  33. Rodney L Summerscales, Shlomo Argamon, Shangda Bai, Jordan Hupert, and Alan Schwartz. 2011. Automatic summarization of results from clinical trials. In 2011 IEEE International Conference on Bioinformatics and Biomedicine, pages 372–377. IEEE. find
  34. Mikhail Tikhomirov, N. Loukachevitch, Anastasiia Sirotina, and Boris Dobrov. 2020. Using bert and augmentation in named entity recognition for cybersecurity domain. In Natural Language Processing and Information Systems, pages 16–24, Cham. Springer International Publishing. find
  35. Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2005. Ace 2005 multilingual training corpus-linguistic data consortium. URL: https://catalog.ldc. upenn. edu/LDC2006T06. ↗
  36. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013. Ontonotes release 5.0 ldc2013t19. Technical report, Linguistic Data Consortium. find
  37. Ziqi Zhang. 2013. Named entity recognition : challenges in document annotation, gazetteer construction and disambiguation. find
  38. Huiming Zhu, Chunhui He, Yang Fang, and Weidong Xiao. 2020. Fine grained named entity recognition via seq2seq framework. IEEE Access, 8:53953– 53961. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liqreina-2023-arabic-fine-grained-entity-recognition/