Elamur Research⌕
☰
home / papers / 2022 / Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT

Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT

Summary

Arabic NER resources had so far been annotated only with flat labels. This means an entity sitting inside another one is lost: a city name inside a university name, or a person's name inside an organization's name. Nested NER also gets little attention in general, mostly because there is not enough data. The authors address this gap with Wojood, a manually annotated corpus of roughly 550K tokens. It covers Modern Standard Arabic and dialectal text from many domains: web articles on health, finance, law, elections, migration and similar topics, plus documents from the Awraq Palestinian historical archive. Annotators labelled 21 entity types and allowed up to four nesting layers. Agreement between annotators was very high.

To show that the data can be learned from, the authors fine-tuned AraBERT in a multi-task setup that predicts the nested tags. It reached a micro F1 of 0.884. Wojood is the first nested NER corpus for Arabic, and it is also larger and broader than earlier flat corpora such as ANERCorp and Ontonotes 5. The corpus, guidelines, code and model are publicly released, and the model is also served as a web service.

Key points

  • Wojood has about 550K tokens and around 75K entity mentions. About 22.5% of these mentions are nested inside another mention, with up to four nesting layers.
  • Among the corpus's 21 entity types are date, event, location, organization and person. It contains both MSA and dialect text, tagged with the IOB2 scheme.
  • The sources are about 258K tokens of web articles (un.org, hrw.org, who.org, quora.com and others) and about 227K tokens from the Awraq archive of Birzeit University.
  • Inter-annotator agreement reached Cohen's Kappa 0.979 and F1 0.976.
  • The multi-task fine-tuned AraBERT model scored a micro F1 of 0.884. The corpus, guidelines, code and model are public.

Interesting ideas

  • I1. Annotating embedded entities rather than flattening them keeps facts such as "this organization is named after a place/person" recoverable from the labels themselves. — For Mova (world): the entity store could keep inner mentions of a name as linked entities, e.g. a GPE inside an ORG, so the world graph gets a free "located in" or "named after" hint.
  • I2. Nested tagging can be treated as several parallel sequence-labelling tasks on one shared pre-trained encoder, which avoids a heavy recursive decoder. — For Mova (tree): gparse's transformer encoder could carry extra cheap tagging heads, such as entity layers, next to the dependency head, so both use one forward pass.
  • I3. Annotation quality is reported twice: as a chance-corrected Kappa and as an F1 over mentions. The second number shows span-level disagreement that the first one can hide. — For Mova (learn): agreement among the witness parsers on silver data could be scored the same way, using both a chance-corrected statistic and an arc/label F1, to decide which trees to trust.
  • I4. A surveyed approach (Ju et al.) stacks flat tagging layers and stops once a layer outputs only "O". This gives a natural, data-driven depth for nested structure. — For Mova (tokens): a named-entity chunker could peel nested names layer by layer with the same stopping rule, keeping each layer explainable.
  • I5. The new corpus is validated by training a model on it and reporting the score, and it is released together with the guidelines and code. — For Mova (learn): every new silver treebank could ship with a "train the parser on it and evaluate" check, plus its written conversion rules, before it is admitted to training.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Named entity recognition

Cited by

53 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 48 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{jarrar2022wojood,
  title = {{Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT}},
  author = {Jarrar, Mustafa and Khalilia, Mohammed and Ghanem, Sana},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
  year = {2022},
  doi = {10.63317/57rz4w3or4jm},
  url = {https://doi.org/10.63317/57rz4w3or4jm}
}

References

63 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdallah, S., Shaalan, K., and Shoaib, M. (2012). Integrating rule-based system with classification for arabic named entity recognition. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 311–322. Springer. find
  2. Abdul-hamid, A. and Darwish, K. (2010). Simplified feature set for arabic named entity recognition. Proceedings of the 2010 Named Entities Workshop. find
  3. Abdul-Mageed, M., Elmadany, A. R., and Nagoudi, E. M. B. (2021). Arbert & marbert: Deep bidirectional transformers for arabic. In Proceedings of the ACL-IJCNLP 2021. ACL. ↗ our card
  4. Abuaiadah, D., Rajendran, D., and Jarrar, M. (2017). Clustering arabic tweets for sentiment analysis. In Proceedings of the 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications, pages 499–506. IEEE Computer Society, November. find
  5. Al-Hajj, M. and Jarrar, M. (2021a). Arabglossbert: Fine-tuning bert on context-gloss pairs for wsd. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 40–48, Online, sep. INCOMA Ltd. find
  6. Al-Hajj, M. and Jarrar, M. (2021b). Lu-bzu at semeval-2021 task 2: Word2vec and lemma2vec performance in arabic word-in-context disambiguation. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 748–755, Online, aug. Association for Computational Linguistics. find
  7. Al-Qurishi, M. and Souissi, R. (2021). Arabic named entity recognition using transformer-based-crf model. In Proceedings of the ICNLSP 2021 Conference. find
  8. Ali, B. A. B., Mihi, S., Bazi, I. E., and Laachfoubi, N. (2020). A recent survey of arabic named entity recognition on social media. Revue d’Intelligence Artificielle, 34. ↗
  9. Antoun, W., Baly, F., and Hajj, H. (2020). Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference, 2. ↗ our card
  10. Benajiba, Y., Rosso, P., and Ruiz, J. M. B. (2007). Anersys: An arabic named entity recognition system based on maximum entropy. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), volume 4394 LNCS. find
  11. Bhatia, P., Celikkaya, B., and Khalilia, M. (2019). Joint entity extraction and assertion detection for clinical text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 954–959, Florence, Italy, July. Association for Computational Linguistics. find
  12. Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. (2017). Enriching Word Vectors with Subword Information. Transactions of the Association for Computational Linguistics, 5:135–146, 06. ↗
  13. Brandsen, A., Verberne, S., Wansleeben, M., and Lambers, K. (2020). Creating a dataset for named entity recognition in the archaeology domain. In LREC 2020 - 12th International Conference on Language Resources and Evaluation, Conference Proceedings. ↗
  14. Campillos-Llanos, L., Valverde-Mateos, A., Capllonch-Carrión, A., and Moreno-Sandoval, A. (2021). A clinical trials corpus annotated with umls entities to enhance the access to evidence-based medicine. BMC Medical Informatics and Decision Making, 21. ↗
  15. Dai, X. and Adel, H. (2020). An analysis of simple data augmentation for named entity recognition. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3861– 3867, Barcelona, Spain (Online), December. International Committee on Computational Linguistics. ↗ our card
  16. Darwish, K., Habash, N., Abbas, M., Al-Khalifa, H., Al-Natsheh, H. T., Bouamor, H., Bouzoubaa, K., Cavalli-Sforza, V., El-Beltagy, S. R., El-Hajj, W., Jarrar, M., and Mubarak, H. (2021). A panoramic survey of natural language processing in the arab worlds. Commun. ACM, 64(4):72–81, April. find
  17. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171– 4186, Minneapolis, Minnesota, June. Association for Computational Linguistics. find
  18. Di Eugenio, B. and Glass, M. (2004). The kappa statistic: A second look. Computational Linguistics, 30(1):95–101. ↗
  19. El-Khair, I. A. (2016). 1.5 billion words arabic corpus. arXiv preprint arXiv:1611.04033. ↗
  20. Feng, X., Qin, B., and Liu, T. (2018). A language-independent neural network for event detection. Science China Information Sciences, 61. ↗ our card
  21. Finkel, J. R. and Manning, C. D. (2009). Nested named entity recognition. In Proceedings of ACL and AFNLP 2009, pages 141–150. ACL. find our card
  22. Guo, J., Xu, G., Cheng, X., and Li, H. (2009). Named entity recognition in query. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 267–274. find
  23. Haff, K. E., Jarrar, M., Hammouda, T., and Zaraket, F. (2022). Curras + baladi: Towards a levantine corpus. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France, June. ↗
  24. Hassan, H. and Sorensen, J. (2005). An integrated approach for arabic-english named entity translation. In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 87– 93. find
  25. Hripcsak, G. and Rothschild, A. S. (2005). Agreement, the f-measure, and reliability in information retrieval. Journal of the American Medical Informatics Association, 12. ↗
  26. Jaber, A. and Zaraket, F. A. (2018). Morphology-based entity and relational entity extraction framework for arabic. Revue Traitement Automatique des Langues, 58. ↗
  27. Jarrar, M. and Amayreh, H. (2019). An arabic-multilingual database with a lexicographic search engine. In The 24th International Conference on Applications of Natural Language to Information Systems (NLDB 2019), volume 11608 of LNCS, pages 234–246. Springer, June. find
  28. Jarrar, M., Deik, A., and Faraj, B. (2011). Ontology-based data and process governance framework -the case of e-government interoperability in palestine. In Proceedings of the IFIP International Symposium on Data-Driven Process Discovery and Analysis (SIMPDA’11), pages 83–98, 6. find
  29. Jarrar, M., Habash, N., Akra, D., and Zalmout, N. (2014). Building a corpus for palestinian arabic: a preliminary study. In Proceedings of the EMNLP 2014, Workshop on Arabic Natural Language, pages 18–27. Association For Computational Linguistics, October. find
  30. Jarrar, M., Habash, N., Alrimawi, F., Akra, D., and Zalmout, N. (2017). Curras: An annotated corpus for the palestinian arabic dialect. Journal Language Resources and Evaluation, 51(3):745–775, September. ↗
  31. Jarrar, M., Amayreh, H., and McCrae, J. P. (2019). Representing arabic lexicons in lemon - a preliminary study. In The 2nd Conference on Language, Data and Knowledge (LDK 2019), volume 2402, pages 29–33. CEUR Workshop Proceedings, May. ↗
  32. Jarrar, M., Karajah, E., Khalifa, M., and Shaalan, K. (2021). Extracting synonyms from bilingual dictionaries. In Proceedings of the 11th International Global Wordnet Conference (GWC2021), pages 215–222. Global Wordnet Association, Jan. find
  33. Jarrar, M. (2011). Building a formal arabic ontology (invited paper). In Proceedings of the Experts Meeting on Arabic Ontologies and Semantic Networks. ALECSO, Arab League, 7. find
  34. Jarrar, M. (2021). The arabic ontology - an arabic wordnet with ontologically clean content. Applied Ontology Journal, 16(1):1–26. ↗ our card
  35. Ju, M., Miwa, M., and Ananiadou, S. (2018). A neural layered model for nested named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1446–1459, New Orleans, Louisiana, June. Association for Computational Linguistics. find our card
  36. Katiyar, A. and Cardie, C. (2018). Nested named entity recognition revisited. In Proceedings of the NAACL HLT 2018, volume 1, pages 861–871. ACL. find our card
  37. Khalifa, M. and Shaalan, K. (2019). Character convolutions for arabic named entity recognition with long short-term memory networks. Computer Speech and Language, 58. ↗
  38. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260–270, San Diego, California, June. Association for Computational Linguistics. ↗
  39. Li, L., Jin, L., Jiang, Z., Song, D., and Huang, D. (2015). Biomedical named entity recognition based on extended recurrent neural networks. In 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 649–652. find
  40. Lu, W. and Roth, D. (2015). Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 857– 867, Lisbon, Portugal, September. Association for Computational Linguistics. find
  41. Martı́nez-deMiguel, C., Segura-Bedmar, I., Chacón-Solano, E., and Guerrero-Aspizua, S. (2022). The raredis corpus: A corpus annotated with rare diseases, their signs and symptoms. Journal of Biomedical Informatics, 125:103961, 1. ↗
  42. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. In Yoshua Bengio et al., editors, 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings. ↗
  43. Mohit, B., Schneider, N., Bhowmick, R., Oflazer, K., and Smith, N. (2012). Recall-oriented learning of named entities in arabic wikipedia. In EACL 2012 Proceedings. find
  44. Mostefa, D., Laı̈b, M., Chaudiron, S., Choukri, K., and Chalendar, G. D. (2009). A multilingual named entity corpus for arabic , english and french named entity tagset the afp corpus. In Proceedings of the Second International Conference on Arabic Language Resources and Tools. find
  45. Ohta, T., Tateisi, Y., and Kim, J. (2002). The genia corpus: An annotated research abstract corpus in molecular biology domain. In Proceedings of the second international conference on Human Language Technology Research, pages 82–86. Morgan Kaufmann Publishers Inc. find
  46. Passos, A., Kumar, V., and McCallum, A. (2014). Lexicon infused phrase embeddings for named entity resolution. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 78–86, Ann Arbor, Michigan, June. Association for Computational Linguistics. find our card
  47. Pennington, J., Socher, R., and Manning, C. (2014). GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, October. Association for Computational Linguistics. find
  48. Ringland, N., Dai, X., Hachey, B., Karimi, S., Paris, C., and Curran, J. R. (2020). Nne: A dataset for nested named entity recognition in english newswire. In Proceedings of ACL 2019. ACL. find our card
  49. Salah, R. E. and Zakaria, L. Q. B. (2018). Building the classical arabic named entity recognition corpus (canercorpus). Journal of Theoretical and Applied Information Technology, 96. find
  50. Sang, E. F. T. K. and Veenstra, J. (1999). Representing text chunks. In Proceedings of the Ninth Conference on European Chapter of the Association for Computational Linguistics, EACL ’99, page 173–179, USA. Association for Computational Linguistics. find our card
  51. Shaalan, K. and Raza, H. (2007). Person name entity recognition for arabic. In Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources, pages 17–24. find
  52. Shaalan, K. (2014). A survey of arabic named entity recognition and classification. Computational Linguistics, 40. ↗
  53. Shaheen, M. and Ezzeldin, A. M. (2014). Arabic question answering: systems, resources, tools, and future trends. Arabian Journal for Science and Engineering, 39(6):4541–4564. ↗
  54. Shibuya, T. and Hovy, E. (2020). Nested Named Entity Recognition via Second-best Sequence Learning and Decoding. Transactions of the Association for Computational Linguistics, 8:605–620, 09. ↗ our card
  55. Sohrab, M. G. and Miwa, M. (2018). Deep exhaustive model for nested named entity recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2843– 2849, Brussels, Belgium, October-November. Association for Computational Linguistics. find our card
  56. Tjong Kim Sang, E. F. and De Meulder, F. (2003). Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142–147. ↗
  57. Tjong Kim Sang, E. F. (2002). Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition. In COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002). ↗ our card
  58. Uzuner, Ö., South, B. R., Shen, S., and DuVall, S. L. (2011). 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552–556. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems, pages 5998– 6008. ↗
  59. Walker, C., Strassel, S., Medero, J., and Maeda, K. (2006). Ace 2005 multilingual training corpus ldc2006t06. Technical report, Linguistic Data Consortium, Philadelphia. find
  60. Wang, J., Shou, L., Chen, K., and Chen, G. (2020). Pyramid: A layered model for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5918–5928, Online, July. Association for Computational Linguistics. find our card
  61. Weischedel, R., Pradhan, S., Ramshaw, L., Palmer, M., Xue, N., Marcus, M., Taylor, A., Greenberg, C., Hovy, E., Belvin, R., et al. (2011). Ontonotes release 4.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium. ↗
  62. Weischedel, R., Palmer, M., Marcus, M., Hovy, E., Pradhan, S., Ramshaw, L., Xue, N., Taylor, A., Kaufman, J., Franchini, M., et al. (2013). Ontonotes release 5.0 ldc2013t19. Technical report, Linguistic Data Consortium. find
  63. Zeroual, I., Goldhahn, D., Eckart, T., and Lakhouaja, A. (2019). OSIAN: Open source international Arabic news corpus - preparation and integration into the CLARIN-infrastructure. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 175–182, Florence, Italy, August. Association for Computational Linguistics. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jarrar-2022-wojood-nested-arabic-named-entity/