Elamur Research⌕
☰
home / papers / 2022 / CAraNER: The COVID-19 Arabic Named Entity Corpus

CAraNER: The COVID-19 Arabic Named Entity Corpus

Paper · 2022 · Abdulmohsen Al-Thubaity, Sakhar Alkhereyf, Wejdan Alzahrani et al.

Summary

The paper argues that Arabic NER resources age quickly. New topics, terms and names keep appearing in news text, a form of data and concept drift, and each application needs its own tagset. To address this for the pandemic era, the authors make two contributions. The first is Wassem, a Django-based web platform for annotating Arabic text at the document, token and subword levels; it supports tasks such as classification, NER, segmentation and diacritization. The second is CAraNER, a NER corpus built with Wassem. It contains 1,278 sentences sampled at random from Saudi newspaper articles in the AraNPCC corpus, covering 2019 through 2021. Five annotators labeled it with Person, Title, Location, Organization and Miscellaneous, plus an outside label. As a benchmark, four BERT-style models were fine-tuned on an 80/20 split. AraBERTv0.2-large scored highest with 0.86 macro F1, and CAMeLBERT-MSA came close despite being a much smaller model. The corpus is free to download and gives Arabic NER a time-anchored news dataset that reflects COVID-era vocabulary and entities.

Key points

  • CAraNER has 55,389 tokens in 1,278 sentences. Only about 7% of tokens (3,813) belong to named entities; the rest are O.
  • Tagset: PER, TIT, LOC, ORG, MIS, in BIO format. ORG makes up about 25.0% of entities, MIS about 21.4% and LOC about 12.9%. MIS covers things like a contact-tracing app, diseases and festivals.
  • Wassem has several labor-saving features:
    • word lists with fixed tags that are applied automatically;
    • automatic majority-vote labels when three or more annotators work on a task;
    • volunteer annotator profiles recording background, used when choosing annotators.
  • Fine-tuning used lr 3e-5 with AdamW, 16 epochs, batch size 4 and max length 295. Macro F1 scores:
    • mBERT: 0.77
    • GigaBERT-v4: 0.80
    • CAMeLBERT-MSA: 0.84
    • AraBERTv0.2-large: 0.86 (371M parameters)
  • Per-tag results follow the same order for every model: PER > TIT > ORG > LOC > MIS. ORG and LOC errors mostly come from wrong entity start boundaries. MIS is weak because its contents are so varied.

Interesting ideas

  • I1. Closed-class or highly frequent words with a fixed label are tagged automatically before humans see the text, so annotator effort goes only to context-dependent tokens.
  • I2. The platform only produces a final label by majority vote once at least three annotators exist, which keeps a single judgment from becoming gold.
  • I3. The authors note that accuracy above 95% means little when 93% of tokens are O, so they compare models by macro F1 and per-tag F1.
  • I4. Titles get their own tag instead of being folded into Person. Honorifics and roles such as "Custodian of the Two Holy Mosques" often stand in for a person, and naming them separately makes that link visible.
  • I5. The corpus is deliberately sampled from a bounded period spanning before and after a disruptive event, so that drift in entities and vocabulary can be studied.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A small hand-checked lexicon of fixed-tag function words could pre-fill tags, leaving tagger disagreement review to the truly ambiguous tokens.
  • I2 (learn): Silver data from witness parsers could adopt the same explicit quorum rule, so a token or arc becomes silver only with enough independent votes, and every rejection can be explained.
  • I3 (learn): Evaluations should always report per-label scores and an all-majority-class baseline as a negative control, so rare-but-important labels are not hidden by frequent easy ones.
  • I4 (frames): Coreference could treat a title mention as an explicit role pointing to a person entity, so that "the Minister" and the named minister can be merged with a stated reason.
  • I5 (world): Test sets for the compiled knowledge could be time-sliced to measure how many new entities fall outside the ontology or knowledge graph, flagging where knowledge has gone stale.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓King Saud UniversityRiyadh · Saudi Arabia37 works · 527 citations🔬King Abdulaziz City for Science and TechnologyRiyadh · Saudi Arabia7 works · 74 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksBiochemistry, Genetics and Molecular Biology · 234 worksMolecular Biology · 231 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 2,908 worksNamed entity recognition · 200 works

Cite

Cite as Abdulmohsen Al-Thubaity, Sakhar Alkhereyf, Wejdan Alzahrani and Alia Bahanshal. 2022. CAraNER: The COVID-19 Arabic Named Entity Corpus. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.wanlp-1.1

@inproceedings{althubaity2022caraner,
  title = {{CAraNER: The COVID-19 Arabic Named Entity Corpus}},
  author = {Abdulmohsen Al-Thubaity, Sakhar Alkhereyf, Wejdan Alzahrani and Alia Bahanshal},
  booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)},
  year = {2022},
  doi = {10.18653/v1/2022.wanlp-1.1},
  url = {https://doi.org/10.18653/v1/2022.wanlp-1.1}
}

References

29 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdulmohsen Al-Thubaity, Sakhar Alkhereyf, and Alia O. Bahanshal. 2022. AraNPCC: The Arabic newspaper covid-19 corpus. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur’an QA and Fine-Grained Hate Speech Detection, pages 32– 40, Marseille, France. European Language Resources Association.
  2. Ismail Harrando and Raphaël Troncy. 2021. Improving media content recommendation with automatic annotations. In KaRS 2021, 3rd Edition of Knowledge-aware and Conversational Recommender Systems. find
  3. Ahmed Aliwy, Ayad Abbas, and Ahmed Alkhayyat. 2021. NERWS: Towards improving information retrieval of digital library management system using named entity recognition and word sense. Big Data and Cognitive Computing, 5(4):59. ↗
  4. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 92– 104.
  5. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. ↗
  6. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. Wojood: Nested Arabic named entity corpus and recognition using BERT. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France, June.
  7. Yassine Benajiba, Paolo Rosso, and José Miguel Benedíruiz. 2007. Anersys: An Arabic named entity recognition system based on maximum entropy. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 143–153. Springer. find
  8. Michal Konkol and Miloslav Konopík. 2013. Crf-based czech named entity recognizer and consolidation of czech ner research. In International conference on text, speech and dialogue, pages 153–160. Springer. find
  9. Bojan Bozic, Jayadeep Kumar Sasikumar, and Tamara Matthews. 2021. KnowText: Auto-generated knowledge graphs for custom domain applications. In The 23rd International Conference on Information Integration and Web Intelligence, pages 350–358. find
  10. Wuwei Lan, Yang Chen, Wei Xu, and Alan Ritter. 2020. Gigabert: Zero-shot transfer learning from English to Arabic. In Proceedings of The 2020 Conference on Empirical Methods on Natural Language Processing (EMNLP). find
  11. Aditya Kiran Brahma, Prathyush Potluri, Meghana Kanapaneni, Sumanth Prabhu, and Sundeep Teki. 2021. Identification of food quality descriptors in customer chat conversations using named entity recognition. In 8th ACM IKDD CODS and 26th COMAD, pages 257–261. find
  12. Chenguang Liu, Yongli Yu, Xingxin Li, and Peng Wang. 2021. Named entity recognition in equipment support field using tri-training algorithm and text information extraction technology. IEEE Access, 9:126728–126734. ↗
  13. Bilge Celik and Joaquin Vanschoren. 2021. Adaptation strategies for automated machine learning on evolving data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9):3067–3078. ↗
  14. Hao Liu, Qinjun Qiu, Liang Wu, Wenjia Li, Bin Wang, and Yuan Zhou. 2022. Few-shot learning for name entity recognition in geological text based on GeoBERT. Earth Science Informatics, pages 1–13. ↗
  15. Hai Leong Chieu and Hwee Tou Ng. 2003. Named entity recognition with a maximum entropy approach. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, pages 160– 163. find
  16. A Uma Maheswari, N Revathy, T Guhan, B Praveen, and R Magesh Kumar. 2022. A systematic bpclstm algorithm for concept drift detection incorporated sentiment mining. In 2022 International Conference on Computer Communication and Informatics (IC-CCI), pages 1–8. IEEE. find
  17. Kareem Darwish and Wei Gao. 2014. Simple effective microblog named entity recognition: Arabic as an example. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2513–2517.
  18. Songzhu Mei, Cong Liu, Qinglin Wang, and Huayou Su. 2022. Model provenance management in mlops pipeline. In 2022 The 8th International Conference on Computing and Data Engineering, pages 45–50. find
  19. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  20. Alexis Mitchell, Stephanie Strassel, Shudong Huang, and Ramez Zakhary. 2005. Ace 2004 multilingual training corpus. Linguistic Data Consortium, Philadelphia, 1:1–1. ↗
  21. Behrang Mohit, Nathan Schneider, Rishav Bhowmick, Kemal Oflazer, and Noah A Smith. 2012. Recall-oriented learning of named entities in Arabic wikipedia. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 162–173.
  22. Ajeet Grewal and Jimmy Lin. 2018. The evolution of content analysis for personalized recommendations at Twitter. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1355–1356. find
  23. Zara Nasar, Syed Waqar Jaffry, and Muhammad Kamran Malik. 2021. Named entity recognition and relation extraction: State-of-the-art. ACM Computing Surveys (CSUR), 54(1):1–39. ↗
  24. Keqin Peng, Chuantao Yin, Wenge Rong, Chenghua Lin, Deyu Zhou, and Zhang Xiong. 2021. Named entity aware transfer learning for biomedical factoid question answering. IEEE/ACM Transactions on Computational Biology and Bioinformatics. ↗
  25. Ramzi Esmail Salah and Lailatul Qadri Binti Zakaria. 2018. Building the Classical Arabic named entity recognition corpus (canercorpus). In 2018 Fourth International Conference on Information Retrieval and Knowledge Management (CAMP), pages 1–8. IEEE. find
  26. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. 2013. Ontonotes release 5.0 ldc2013t19. Linguistic Data Consortium, Philadelphia, PA, 23. find
  27. Gezheng Xu, Wenge Rong, Yanmeng Wang, Yuanxin Ouyang, and Zhang Xiong. 2021. External features enriched model for biomedical question answering. BMC bioinformatics, 22(1):1–19. ↗
  28. Hailemariam Mehari Yohannes and Toshiyuki Amagasa. 2022. Named-entity recognition for a low-resource language using pre-trained language model. In Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing, pages 837–844. find
  29. Yue Zhang and Jie Yang. 2018. Chinese ner using lattice LSTM. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1554–1564.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/thubaity-2022-caraner-covid-19-arabic-named/