Elamur Research⌕
☰
home / papers / 2023 / El-Kawaref at WojoodNER shared task: StagedNER for Arabic Named Entity Recognition

El-Kawaref at WojoodNER shared task: StagedNER for Arabic Named Entity Recognition

Paper · 2023 · Nehal Elkaref and Mohab Elkaref

Summary

The paper describes a system for flat Arabic named entity recognition in the WojoodNER 2023 shared task. Arabic NER is hard because the language is morphologically rich and agglutinative, and it has no capitalisation cues. The authors apply StagedNER, which splits the task between two separate AraBERTv02 models. The first model predicts only BIO boundary tags, with help from POS tags produced by CAMeL Tools (MSA or Levant taggers). The second, independently fine-tuned model predicts only entity classes. It uses the boundaries to pool vectors at each entity's start and end.

The data is the Wojood corpus: over 550K tokens, about 86% Modern Standard Arabic and the rest Levantine dialect. Three base models (MSA POS, LEV POS, no POS) were combined into four ensembles. The best ensemble paired the MSA-POS model with the no-POS model. It reached 90.03% F1 on the development set and 91.95% on the test set. This placed second in F1, fourth in precision and first in recall. The result shows that factoring NER into boundaries and types is competitive with joint tagging.

Key points

  • Corpus: Wojood, with only its flat-entity annotations used here.
    • MSA text comes from the Birzeit "Awraq" archive and web articles.
    • Dialect text comes from Lebanese and Palestinian corpora.
    • O tokens dominate (about 258K in train, 36K in validation).
  • Architecture: two AraBERTv02 instances.
    • Stage 1 (BIO) inserts POS tags as special tokens between input tokens.
    • Stage 2 (entity type) pools the start and end vectors of each span.
    • Gold BIO tags were used in training and validation; stage-1 predictions were used on test.
  • Best test result came from the MSA POS + No POS ensemble: F1 91.95%, P 91.43%, R 92.48%.
    • The task winner (LIPN) scored 91.96% F1.
    • The dev baseline F1 was 86.81%.
  • Training setup: learning rate 5e-5, batch size 8, dropout 0.1, 8 or 16 epochs. Training took about 11 minutes per epoch on an A100 80GB.
  • Per-class results were weak on WEBSITE (about 54% F1) and PRODUCT (about 44%), and zero on CURR and PERCENT. The authors trace these errors to dialectal or non-Arabic strings.

Interesting ideas

  • I1. Separate "where is the span" from "what kind is it" using two independent models, so the type classifier sees only bare classes (ORG rather than B-ORG/I-ORG) and the label space shrinks.
  • I2. Inject auxiliary linguistic features (here POS) as special tokens interleaved with the input, rather than through extra embedding layers, so a pretrained transformer can use them without architectural changes.
  • I3. Classify a span from pooled vectors at its first and last positions, using the predicted boundaries as the pooling guide.
  • I4. Build ensembles from variants that differ only in the auxiliary signal (MSA POS, dialect POS, none), and treat which combination wins as evidence about how useful each signal is.
  • I5. Training the second stage on gold boundaries but testing it on predicted ones exposes a train/test mismatch worth measuring explicitly.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Entity mention detection could first fix span boundaries from the UD tree, then run a separate typer over those spans for the ontology, so each step's error can be explained on its own.
  • I2 (tree): gparse could receive tagger output or morphology as inline marker tokens, and an ablation would show whether the parser actually relies on them.
  • I3 (frames): Semantic role spans for VerbNet frames could be represented by their edge vectors, which gives a cheap, inspectable span representation.
  • I4 (learn): Witness parsers for silver data could differ by input features, and their agreement patterns could serve as a diagnostic of which features matter.
  • I5 (learn): Evaluation could report pipeline stages both with gold upstream input and with predicted upstream input, to quantify how errors propagate between stages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓German University in CairoCairo · Egypt6 works · 74 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksNamed entity recognition · 200 works

Cite

Cite as Nehal Elkaref and Mohab Elkaref. 2023. El-Kawaref at WojoodNER shared task: StagedNER for Arabic Named Entity Recognition. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2023.arabicnlp-1.91

@inproceedings{elkaref2023el,
  title = {{El-Kawaref at WojoodNER shared task: StagedNER for Arabic Named Entity Recognition}},
  author = {Nehal Elkaref and Mohab Elkaref},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.91},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.91}
}

References

19 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Samir AbdelRahman, Mohamed Elarnaoty, Marwa Magdy, and Aly Fahmy. 2010. Integrated machine learning techniques for arabic named entity recognition. IJCSI, 7(4):27–36. find
  2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. ARBERT & MARBERT: Deep bidirectional transformers for Arabic. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. ↗
  3. Muhammad Saleh Al-Qurishi and Riad Souissi. 2021. Arabic named entity recognition using transformer-based-crf model. In Proceedings of the 4th Interna-4 https://huggingface.co/spaces/nehalelkaref/ flat-arabic-entity-classification tional Conference on Natural Language and Speech Processing (ICNLSP 2021), pages 262–271. ↗
  4. Norah Alsaaran and Maha Alrabiah. 2021. Classical arabic named entity recognition using variant deep neural network architectures and bert. IEEE Access, 9:91537–91547. ↗
  5. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020a. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. ↗
  6. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020b. Araelectra: Pre-training text discriminators for arabic language understanding. arXiv preprint arXiv:2012.15516. ↗
  7. Qiao Cheng, Juntao Liu, Xiaoye Qu, Jin Zhao, Jiaqing Liang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan, and Yanghua Xiao. 2021. Hacred: A large-scale relation extraction dataset toward hard cases in practical applications. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2819–2831. find
  8. Kevin Clark and Christopher D Manning. 2016. Improving coreference resolution by learning entity-level distributed representations. arXiv preprint arXiv:1606.01323. ↗
  9. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. CoRR, abs/1911.02116. ↗
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  11. Mohab Elkaref, Nathan Herr, Shinnosuke Tanaka, and Geeth De Mel. 2023. NLPeople at SemEval-2023 task 2: A staged approach for multilingual named entity recognition. In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1148–1153, Toronto, Canada. Association for Computational Linguistics. find
  12. Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras+ baladi: Towards a levantine corpus. arXiv preprint arXiv:2205.09692. ↗
  13. Mustafa Jarrar, Muhammad Abdul-Mageed, Mohammed Khalilia, Bashar Talafha, AbdelRahim Elmadany, Nagham Hamad, and Alaa Omer. 2023. WojoodNER 2023: The First Arabic Named Entity Recognition Shared Task. In Proceedings of the First Arabic Natural Language Processing Conference (ArabicNLP 2023).
  14. Mustafa Jarrar, Nizar Habash, Diyam Akra, and Nasser Zalmout. 2014. Building a corpus for palestinian arabic: a preliminary study. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 18–27.
  15. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: an annotated corpus for the palestinian arabic dialect. Language Resources and Evaluation, 51:745–775. ↗
  16. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested Arabic named entity corpus and recognition using BERT. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 3626–3636, Marseille, France. European Language Resources Association.
  17. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association.
  18. Xiaoye Qu, Yingjie Gu, Qingrong Xia, Zechang Li, Zhefeng Wang, and Baoxing Huai. 2023. A survey on arabic named entity recognition: Past, recent advances, and future trends. arXiv preprint arXiv:2302.03512. ↗
  19. Arata Ugawa, Akihiro Tamura, Takashi Ninomiya, Hiroya Takamura, and Manabu Okumura. 2018. Neural machine translation incorporating named entity. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3240–3250. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elkaref-2023-el-kawaref-wojoodner-shared-task/