Elamur Research⌕
☰
home / papers / 2025 / Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition

Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition

Paper · 2025 · Nagham Hamad, Mohammed Khalilia and Mustafa Jarrar

Summary

Arabic NER systems perform well on Modern Standard Arabic (MSA) news, but they are seldom tested on dialects or on other topics, because suitable labelled data hardly exists. The authors built Konooz to fill this gap. It is a grid of 160 small corpora, one for each pairing of 16 varieties (15 dialects plus MSA) with 10 domains such as politics, finance, law, health, art and sport. The texts are public social-media posts and comments, with MSA articles taken from news sites. Annotation follows the Wojood guidelines, with 21 entity types in both flat and nested form, and was refined over several model-assisted review rounds.

The authors used Konooz to test four existing Arabic NER models (Wojood nested/flat, OntoNotes- and ANERCorp-trained). Scores fell by up to 38% compared with in-distribution test data. To explain the losses, they measured how far each test cell lies from the training data using Maximum Mean Discrepancy (MMD). The resource is open and supports domain-adaptation and transfer research on low-resource varieties.

Key points

  • Konooz has about 777k tokens. Each dialect–domain cell holds roughly 4k tokens, and MSA cells hold about 8k. Sentences were written between 2010 and 2022.
  • Collection and checking:
    • 40 students collected the data after first watching about two hours of content in their target dialect.
    • In a blind dialect-identification test with mixed similar dialects, native speakers were right 87% of the time on average.
    • Sentences with an ALDi score below 0.2 were checked by hand, and 8% of sentences were removed as actually MSA.
  • Annotation ran in three phases:
    • First, the Wojood model in SinaTools produced pre-tags.
    • Next, five linguists with master's degrees annotated over 15 months.
    • Finally, a model retrained on Wojood, Wojood Gaza and the first Konooz version re-tagged the data. Reviewers fixed about 1,500 errors, and a further round found only 10 changes.
  • Out-of-domain performance fell by 29–38%. Wojood Nested was best at 63% and scored highest on history, the domain with the lowest MMD to Wojood.
  • Entity coverage mattered more than lexical similarity:
    • ANERCorp scored best on Egyptian (F1 66%).
    • OntoNotes scored best on Lebanese (F1 51%), helped by its 2,099 Lebanon-related entities.
    • Wojood handled Iraqi text fairly well despite low lexical similarity, because Iraq-related phrases appear more than 700 times in its training data.

Interesting ideas

  • I1. The corpus is a balanced factorial grid, with roughly equal size for every domain × dialect cell, so the effects of topic shift and variety shift can be separated rather than mixed in one test set.
  • I2. The annotation loop retrains a model on the current labels, re-tags everything, and lets humans review the output; the shrinking number of corrections (1,500, then 10) serves as a convergence signal.
  • I3. A cheap continuous score (ALDi) filters the data automatically, and humans review only the items below a threshold, which keeps manual effort focused where contamination is likely.
  • I4. Training–test divergence (MMD) partly explains score drops, but the strongest predictor turned out to be whether the test entities were already seen in training, not overall vocabulary overlap.
  • I5. Data validity was checked by mixing samples of easily confused dialects and asking judges to tell them apart.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Build evaluation sets as a grid of genre × register (fables, school texts, word problems, news), so that a parser or reader regression can be traced to one factor.
  • I2 (learn): Use the falling count of silver-data edits across witness-parser retraining rounds as an explicit stopping rule for repair iterations.
  • I3 (tree): Score sentences by distance from standard UD English conventions and send only low-confidence ones for review during treebank or dialect conversion.
  • I4 (world): Before evaluating on new texts, measure how many of their entities already exist in the knowledge graph, and report this coverage alongside accuracy.
  • I5 (learn): Add confusable negative-control items, such as near-paraphrases with swapped roles, to evaluations, so that a model's apparent skill can be confirmed as discrimination rather than surface matching.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksNamed entity recognition · 200 works

Cite

Cite as Nagham Hamad, Mohammed Khalilia and Mustafa Jarrar. 2025. Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition. Findings of the Association for Computational Linguistics: ACL 2025.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2025.findings-acl.382

@inproceedings{hamad2025konooz,
  title = {{Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition}},
  author = {Nagham Hamad, Mohammed Khalilia and Mustafa Jarrar},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
  year = {2025},
  doi = {10.18653/v1/2025.findings-acl.382},
  url = {https://doi.org/10.18653/v1/2025.findings-acl.382}
}

References

48 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Moustafa Al-Hajj and Mustafa Jarrar. 2021. Arab-GlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 40–48, Online. INCOMA Ltd. find
  2. Alaa Aljabari, Lina Duaibes, Mustafa Jarrar, and Mohammed Khalilia. 2024. Event-Arguments Extraction Corpus and Modeling using BERT for Arabic. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  3. Wissam Antoun, Fady Baly, and Hazem M. Hajj. 2020. Arabert: Transformer-based model for arabic language understanding. CoRR, abs/2003.00104. ↗
  4. Osama Badawy, Mohamed Shaheen, and Abdelbaki Hamadene. 2011. Arqa: An intelligent arabic question answering system. In Proceedings of Arabic Language Technology International Conference (ALTIC 2011), pages 1–8. find
  5. Sylvio Barbon Junior, Paolo Ceravolo, Sven Groppe, Mustafa Jarrar, Samira Maghool, Florence Sèdes, Soror Sahri, and Maurice Van Keulen. 2024. Are Large Language Models the New Interface for Data Pipelines? In Proceedings of the International Workshop on Big Data in Emergent Distributed Environments, BiDEDE ’24, New York, NY, USA. Association for Computing Machinery. ↗
  6. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Mach. Learn., 79(1-2):151–175. ↗
  7. Yassine Benajiba, Paolo Rosso, and José Miguel Benedíruiz. 2007. Anersys: An arabic named entity recognition system based on maximum entropy. In Computational Linguistics and Intelligent Text Processing: 8th International Conference, CICLing 2007, Mexico City, Mexico, February 18-24, 2007. Proceedings 8, pages 143–153. Springer. find
  8. Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46. ↗
  9. Abdelhalim Hafedh Dahou and Mohamed Amine Cheragui. 2023. Dzner: A large algerian named entity recognition dataset. Natural Language Processing Journal, 3:100005. ↗
  10. Xiang Dai, Sarvnaz Karimi, Ben Hachey, and Cécile Paris. 2019. Using similarity measures to select pretraining data for NER. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 1460– 1470. Association for Computational Linguistics. ↗
  11. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T. Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R. El-Beltagy, Wassim El-Hajj, Mustafa Jarrar, and Hamdy Mubarak. 2021. A Panoramic survey of Natural Language Processing in the Arab Worlds. Commun. ACM, 64(4):72–81. find
  12. Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander J. Smola. 2008. A kernel method for the two-sample problem. CoRR, abs/0805.2368. ↗
  13. Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras + Baladi: Towards a Levantine Corpus. In Proceedings of the International Conference on Language Resources and Evaluation(LREC 2022), Marseille, France. ↗
  14. Tymaa Hammouda, Mustafa Jarrar, and Mohammed Khalilia. 2024. SinaTools: Open Source Toolkit for Arabic Natural Language Understanding. In Proceedings of the 2024 AI in Computational Linguistics (ACLING 2024), Procedia Computer Science, Dubai. ELSEVIER. find
  15. Hany Hassan and Jeffrey Sorensen. 2005. An integrated approach for arabic-english named entity translation. In Proceedings of the Workshop on Computational Approaches to Semitic Languages, SEMITIC@ACL 2005, Ann Arbor, MI, USA, June 29, 2005, pages 87–93. Association for Computational Linguistics. find
  16. George Hripcsak and Adam S. Rothschild. 2005. Technical brief: Agreement, the f-measure, and reliability in information retrieval. J. Am. Medical Informatics Assoc., 12(3):296–298. find
  17. Mustafa Jarrar. 2021. The Arabic Ontology - An Arabic Wordnet with Ontologically Clean Content. Applied Ontology Journal, 16(1):1–26. ↗
  18. Mustafa Jarrar, Muhammad Abdul-Mageed, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, Nagham Hamad, and Alaa’ Omar. 2023a. WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 748– 758. ACL.
  19. Mustafa Jarrar, Anton Deik, and Bilal Faraj. 2011. Ontology-based data and process governance framework -the case of e-government interoperability in palestine. In Proceedings of the IFIP International Symposium on Data-Driven Process Discovery and Analysis (SIMPDA’11), pages 83–98. find
  20. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: An annotated corpus for the palestinian arabic dialect. Journal Language Resources and Evaluation, 51(3):2-s2.0-85001544989. ↗
  21. Mustafa Jarrar, Nizar Habash, Mo El-Haj, Amal Haddad Haddad, Zeina Jallad, Camille Mansour, Diana Allan, Paul Rayson, Tymaa Hammouda, and Sanad Malaysha, editors. 2025. Proceedings of the first International Workshop on Nakba Narratives as Language Resources. Association for Computational Linguistics, Abu Dhabi, UAE. find
  22. Mustafa Jarrar, Nagham Hamad, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, and Muhammad Abdul-Mageed. 2024. WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  23. Mustafa Jarrar and Tymaa Hasanain Hammouda. 2024. Qabas: An Open-Source Arabic Lexicographic Database. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13363–13370, Torino, Italy. ELRA and ICCL.
  24. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
  25. Mustafa Jarrar, Sanad Malaysha, Tymaa Hammouda, and Mohammed Khalilia. 2023b. SALMA: Arabic Sense-annotated Corpus and WSD Benchmarks. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 359–369. ACL.
  26. Mustafa Jarrar, Fadi Zaraket, Tymaa Hammouda, Daanish Masood Alavi, and Martin Waahlisch. 2023c. Lisan: Yemeni, Irqi, Libyan, and Sudanese Arabic Dialect Copora with Morphological Annotations. In The 20th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA), pages 1–7. IEEE. find
  27. Chen Jia, Liang Xiao, and Yue Zhang. 2019. Cross-domain NER using cross-domain language modeling. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 2464–2474. Association for Computational Linguistics. find
  28. Abhinav Ramesh Kashyap, Devamanyu Hazarika, Min-Yen Kan, and Roger Zimmermann. 2021. Domain divergences: A survey and empirical analysis. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 1830–1849. Association for Computational Linguistics. find
  29. Amr Keleg, Sharon Goldwater, and Walid Magdy. 2023. Aldi: Quantifying the arabic level of dialectness of text. arXiv preprint arXiv:2310.13747. ↗
  30. Mohammed Khalilia, Sanad Malaysha, Reem Suwaileh, Mustafa Jarrar, Alaa Aljabari, Tamer Elsayed, and Imed Zitouni. 2024. ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task. In Proceedings of the Second Arabic Natural Language Processing Conference (ArabicNLP 2024), Bangkok, Thailand. Association for Computational Linguistics.
  31. Niama El Khbir, Urchade Zaratiana, Nadi Tomeh, and Thierry Charnois. 2023. Cross-dialectal named entity recognition in arabic. In Proceedings of ArabicNLP 2023, Singapore (Hybrid), December 7, 2023, pages 140–149. Association for Computational Linguistics.
  32. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  33. Haneen Liqreina, Mustafa Jarrar, Mohammed Khalilia, Ahmed Oumar El-Shangiti, and Muhammad Abdul-Mageed. 2023. Arabic Fine-Grained Entity Recognition. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 310–323. ACL.
  34. Abdellah El Mekki, Abdelkader El Mahdaouy, Ismail Berrada, and Ahmed Khoumsi. 2022. Adasl: An unsupervised domain adaptation framework for arabic multi-dialectal sequence labeling. Inf. Process. Manag., 59(4):102964. ↗
  35. Hanane Nour Moussa and Asmaa Mourhir. 2023. Darnercorp: An annotated named entity recognition dataset in the moroccan dialect. Data in Brief, 48:109234. ↗
  36. Amal Nayouf, Mustafa Jarrar, Fadi zaraket, Tymaa Hammouda, and Mohamad-Bassam Kurdy. 2023. Nâbra: Syrian Arabic Dialects with Morphological Annotations. In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 12–23. ACL.
  37. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. Camel tools: An open source python toolkit for arabic natural language processing. In Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020, pages 7022–7032. European Language Resources Association.
  38. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc. ↗
  39. Siyao Peng, Zihang Sun, Huangyan Shan, Marie Kolm, Verena Blaschke, Ekaterina Artemova, and Barbara Plank. 2024. Sebastian, basti, wastl?! recognizing named entities in bavarian dialectal data. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, pages 14478–14493. ELRA and ICCL. find
  40. Nick Pogrebnyakov and Shohreh Shaghaghian. 2021. Predicting the success of domain adaptation in text similarity. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 206–212, Online. Association for Computational Linguistics. ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. ↗
  41. Peeyush Singhal, Rahee Walambe, Sheela Ramanna, and Ketan Kotecha. 2023. Domain adaptation: Challenges, methods, datasets, and applications. IEEE Access. ↗
  42. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929–1958. find
  43. Bashar Talafha, Karima Kadaoui, Samar Mohamed Magdy, Mariem Habiboullah, Chafei Mohamed Chafei, Ahmed Oumar El-Shangiti, Hiba Zayed, Mohamedou Cheikh Tourad, Rahaf Alhamouri, Rwaa Assi, Aisha Alraeesi, Hour Mohamed, Fakhraddin Alwajih, Abdelrahman Mohamed, Abdellah El Mekki, El Moatez Billah Nagoudi, Benelhadj Djelloul Mama Saadia, Hamzah A. Alsayadi, Walid Al-Dhabyani, Sara Shatnawi, Yasir Ech-chammakhy, Amal Makouar, Yousra Berrachedi, Mustafa Jarrar, Shady Shehata, Ismail Berrada, and Muhammad Abdul-Mageed. 2024. Casablanca: Data and Models for Multidialectal Arabic Speech Recognition. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Miami, Florida, USA. Association for Computational Linguistics. ↗
  44. Samia Touileb. 2022. Nerdz: A preliminary dataset of named entities for algerian. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, AACL/IJCNLP 2022 - Volume 2: Short Papers, Online only, November 20-23, 2022, pages 95–101. Association for Computational Linguistics. ↗
  45. Sowmya Vajjala and Ramya Balasubramaniam. 2022. What do we really know about state of the art NER? In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5983– 5993, Marseille, France. European Language Resources Association. find
  46. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2013a. Ontonotes release 5.0 ldc2013t19. In Technical report, Linguistic Data Consortium. find
  47. Ralph Weischedel, Sameer Pradhan, Lance Ramshaw, Martha Palmer, Nianwen Xue, Mitchell Marcus, Ann Taylor, Craig Greenberg, Eduard Hovy, Robert Belvin, et al. 2013b. Ontonotes release 5.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium. ↗
  48. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-Ayah Zirikly and Mona T. Diab. 2014. Named entity recognition system for dialectal arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing, ANLP@EMNLP 20104, Doha, Qatar, October 25, 2014, pages 78–86. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hamad-2025-konooz-multi-domain-multi-dialect/