Elamur Research⌕
☰
home / papers / 2024 / SinaTools: Open Source Toolkit for Arabic Natural Language Processing

SinaTools: Open Source Toolkit for Arabic Natural Language Processing

Summary

Arabic NLP lacks freely available, easy-to-use tools for semantic and understanding tasks. SinaLab at Birzeit University addresses this with SinaTools, an open-source Python package. It bundles NER (flat, nested and fine-grained), WSD, sentence-level semantic relatedness, synonym extraction and evaluation, lemmatization, POS tagging and root tagging, plus text utilities. Every module is available through an API, a CLI and demo pages.

The approach varies by module. Morphology uses a large frequency-ranked dictionary built from the Qabas lexicographic graph and other SinaLab lexicons, with a BERT fallback for unknown words. NER uses a BERT model fine-tuned on Wojood. Relatedness uses unsupervised cosine similarity over BERT sentence embeddings. Synonymy comes from a graph built from about 100 lexicons.

The authors compare SinaTools with MADAMIRA, CAMeLTools and Farasa. On the ATB and Salma corpora it matches or beats them on lemmatization and POS tagging while running far faster. It also reports top results on NER, WSD and the SemEval-2024 relatedness task. The package gives Arabic researchers and developers a single, practical entry point for language-understanding pipelines.

Key points

  • Headline scores: Flat NER 87.33%, Nested NER 89.42%, WSD 82.63% (micro average on Salma, 34,253 tokens), lemmatization 90.5% on Salma, and POS 97.5% with the coarse tag set.
  • Morphology is the Alma tagger. It maps each wordform to a <lemma, POS, root> solution and always picks the most frequent one, ignoring context. It is built mainly on Qabas, which has about 58k lemmas linked to the Arabic Ontology and 110 lexicons.
  • Processing the ATB took 11.34 s with SinaTools, versus 33.22 s for Farasa, 1661.3 s for MADAMIRA and 13080.2 s for CAMeLTools. On the 40-tag POS set, SinaTools scored 92.7% against at most 82.7% for the others.
  • The NER model, trained on Wojood, covers 21 entity types and stacks one classification layer per type, which is how it handles nested entities.
  • Semantic relatedness reaches a Spearman correlation of 0.49, against a 0.42 baseline, on 595 SemEval-2024 Task 1 test pairs. This was the top-ranked entry in that shared task.

Interesting ideas

  • I1. A precomputed wordform-to-analysis table that always returns the most frequent analysis beat context-aware taggers on accuracy and was orders of magnitude faster, with a neural model used only for unknown words. — For Mova (tokens): build a frequency-ranked wordform→(lemma, UPOS, feats) hashmap from treebanks as the default path in crate en, and call the statistical tagger only for out-of-vocabulary or genuinely ambiguous forms.
  • I2. Nested entities become tractable when each entity type gets its own independent tagging layer, so one token can carry several labels at different depths at once. — For Mova (world): use per-type mention layers so that, for example, a person inside an organization name inside a location yields separate, explainable entity nodes for coreference and the world model.
  • I3. Synonyms can be derived from a graph of synonymy and translation links across many lexicons, where words on a shared cycle count as synonyms with a fuzzy degree, and supplying several seed words sharpens the result. — For Mova (world): apply the cycle idea to WordNet and Wikidata multilingual labels to propose graded synonym sets, which also helps the meta-world translation path.
  • I4. The benchmark scores lemmas both exactly and with sense-numbering stripped, and it times each tool over repeated runs with the first warm-up run discarded. — For Mova (learn): report lemma/sense metrics at two strictness levels and adopt warm-up-excluded averaged timing when comparing the perceptron parser with gparse.
  • I5. Words are compared with diacritic-aware matching, so a partially marked form counts as compatible with a fully marked one unless their explicit marks conflict. — For Mova (tokens): reuse this compatibility-not-equality check for underspecified morphological features, for example when unifying feats from different taggers or treebank dialects.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Named entity recognition · Word sense disambiguation · Part-of-speech tagging

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2411.01523 · arXiv 2411.01523

@misc{hammouda2024sinatools,
  title = {{SinaTools: Open Source Toolkit for Arabic Natural Language Processing}},
  author = {Hammouda, Tymaa and Jarrar, Mustafa and Khalilia, Mohammed},
  year = {2024},
  doi = {10.48550/arxiv.2411.01523},
  eprint = {2411.01523},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2411.01523}
}

References

51 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdalla, M., Vishnubhotla, K., Mohammad, S.M., 2023. What Makes Sentences Semantically Related? A textual relatedness Dataset and Empirical Study, in: Proceedings of EACL 2023, ACL. pp. 782–796. find
  2. Abdul-Mageed, M., Zhang, C., Hashemi, A., et al., 2020. AraNet: A Deep Learning Toolkit for Arabic Social Media, in: OSACT, pp. 16–23. find
  3. Al-Hajj, M., Jarrar, M., 2021a. ArabGlossBERT: Fine-Tuning BERT on Context-Gloss Pairs for WSD., in: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), INCOMA Ltd., Online. pp. 40–48. find
  4. Al-Hajj, M., Jarrar, M., 2021b. LU-BZU at SemEval-2021 Task 2: Word2Vec and Lemma2vec Performance in Arabic Word-in-Context Disambiguation., in: Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), ACL, Online. pp. 748–755. find
  5. Alhafi, D., Deik, A., Jarrar, M., 2019. Usability evaluation of lexicographic e-services, in: The 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), IEE. pp. 1–7. find
  6. Aljabari, A., Duaibes, L., Jarrar, M., Khalilia, M., 2024. Event-Arguments Extraction Corpus and Modeling using BERT for Arabic, in: Proceedings of ArabicNLP’2024, ACL, Bangkok, Thailand. find
  7. Antoun, W., Baly, F., Hajj, H., 2020. AraBERT: Transformer-based Model for Arabic Language Understanding, in: OSACT, pp. 9–15. find our card
  8. Asaadi, S., Mohammad, S.M., Kiritchenko, S., 2019. Big BiRD: A Large, Fine-grained, Bigram Relatedness Dataset for Examining Semantic Composition, in: Proceedings of the NAACL-HLT, ACL. pp. 505–516. find
  9. Darwish, K., Habash, N., Abbas, M., Al-Khalifa, H., Al-Natsheh, H.T., Bouamor, H., Bouzoubaa, K., Cavalli-Sforza, V., El-Beltagy, S.R., El-Hajj, W., Jarrar, M., Mubarak, H., 2021. A Panoramic Survey of Natural Language Processing in the Arab Worlds. Com. ACM 64, 72–81. find
  10. Darwish, K., Mubarak, H., 2016. Farasa: A New Fast and Accurate Arabic Word Segmenter, in: Proceedings of LREC, pp. 1070–1074. find
  11. Duaibes, L., Jaber, A., Jarrar, M., Qadi, A., Qandeel, M., 2024. Sina at FigNews 2024: Multilingual Datasets Annotated with Bias and Propaganda, in: Proceedings of ArabicNLP’2024, ACL, Bangkok, Thailand. find
  12. Elmadany, A., Nagoudi, E.M.B., Abdul-Mageed, M., 2023. Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generation . find
  13. Farha, I.A., Magdy, W., 2019. Mazajak: An Online Arabic Sentiment Analyser, in: Proceedings of WANLP Workshop, pp. 192–198. find
  14. Ghanem, S., Jarrar, M., Jarrar, R., Bounhas, I., 2023. A Benchmark and Scoring Algorithm for Enriching Arabic Synonyms, in: Proceedings of the 12th International Global Wordnet Conference (GWC2023), Global Wordnet Association. pp. 215–222. 6 Demo Page (Diacritic-Based Matching) : https://sina.birzeit.edu/resources/Implication.html ↗
  15. Haff, K.E., Jarrar, M., Hammouda, T., Zaraket, F., 2022. Curras + Baladi: Towards a Levantine Corpus, in: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France. find
  16. Hamad, N., Jarrar, M., Khalilia, M., Nashif, N., 2023. Offensive Hebrew Corpus and Detection using BERT, in: Proceedings of the 20th ACS/IEEE AICCSA, IEEE. find our card
  17. Helou, M.A., Palmonari, M., Jarrar, M., 2016. Effectiveness of automatic translations for cross-lingual ontology mapping. Journal of Artificial Intelligence Research 55, 165–208. find
  18. Jarrar, M., 2008. Towards Effectiveness and Transparency in e-Business Transactions, An Ontology for Customer Complaint Management. IGI Global. chapter 7. pp. 127–149. find
  19. Jarrar, M., 2011. Building a formal arabic ontology (invited paper), in: Proceedings of the Experts Meeting on Arabic Ontologies and Semantic Networks, ALECSO, Arab League. find
  20. Jarrar, M., 2020. Digitization of Arabic Lexicons. UAE Ministry of Culture and Youth. pp. 214–217. find
  21. Jarrar, M., 2021. The Arabic Ontology - An Arabic Wordnet with Ontologically Clean Content. Applied Ontology Journal 16, 1–26. find our card
  22. Jarrar, M., Akra, D., Hammouda, T., 2024a. ALMA: Fast Lemmatizer and POS Tagger for Arabic, in: Proceedings of the 2024 AI in Computational Linguistics (ACLING 2024), ELSEVIER, Dubai. find
  23. Jarrar, M., Amayreh, H., 2019. An arabic-multilingual database with a lexicographic search engine, in: The 24th International Conference on Applications of Natural Language to Information Systems (NLDB 2019), Springer. pp. 234–246. find
  24. Jarrar, M., Amayreh, H., McCrae, J.P., 2019. Representing arabic lexicons in lemon - a preliminary study, in: The 2nd Conference on Language, Data and Knowledge (LDK 2019), CEUR Workshop Proceedings. pp. 29–33. find
  25. Jarrar, M., Birim, A., Khalilia, M., Erden, M., Ghanem, S., 2023a. ArBanking77: Intent Detection Neural Model and a New Dataset in Modern and Dialectical Arabic, in: Proceedings of ArabicNLP, Part of the EMNLP 2023, ACL. pp. 276–287. find
  26. Jarrar, M., Deik, A., Faraj, B., 2011. Ontology-based data and process governance framework -the case of e-government interoperability in palestine, in: Proceedings of the IFIP International Symposium on Data-Driven Process Discovery and Analysis (SIMPDA’11), pp. 83–98. find
  27. Jarrar, M., Habash, N., Akra, D., Zalmout, N., 2014. Building a corpus for palestinian arabic: a preliminary study, in: Proceedings of the EMNLP 2014, Workshop on Arabic Natural Language, ACL. pp. 18–27. find
  28. Jarrar, M., Habash, N., Alrimawi, F., Akra, D., Zalmout, N., 2017. Curras: An annotated corpus for the palestinian arabic dialect. Journal Language Resources and Evaluation 51, 745–775. find
  29. Jarrar, M., Hamad, N., Khalilia, M., Talafha, B., Elmadany, A., Abdul-Mageed, M., 2024b. WojoodNER 2024: The Second Arabic Named Entity Recognition Shared Task, in: Proceedings of ArabicNLP’2024, ACL, Bangkok, Thailand. find
  30. Jarrar, M., Hammouda, T.H., 2024. Qabas: An Open-Source Arabic Lexicographic Database, in: Proceedings of LREC-COLING 2024, ELRA and ICCL, Torino, Italy. pp. 13363–13370. find
  31. Jarrar, M., Karajah, E., Khalifa, M., Shaalan, K., 2021. Extracting Synonyms from Bilingual Dictionaries, in: Proceedings of the 11th International Global Wordnet Conference (GWC2021), Global Wordnet Association. pp. 215–222. find
  32. Jarrar, M., Khalilia, M., Ghanem, S., 2022. Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT, in: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France. find our card
  33. Jarrar, M., Malaysha, S., Hammouda, T., Khalilia, M., 2023b. SALMA: Arabic Sense-annotated Corpus and WSD Benchmarks, in: Proceedings of ArabicNLP, Part of the EMNLP 2023, ACL. pp. 359–369. find
  34. Jarrar, M., Zaraket, F., Asia, R., Amayreh, H., 2018. Diacritic-based matching of arabic words. ACM Asian and Low-Resource Language Information Processing 18, 10:1–10:21. find
  35. Jarrar, M., Zaraket, F., Hammouda, T., Alavi, D.M., Waahlisch, M., 2023c. Lisan: Yemeni, Irqi, Libyan, and Sudanese Arabic Dialect Copora with Morphological Annotations, in: Proceedings of the 20th ACS/IEEE AICCSA, IEEE. find
  36. Khalilia, M., Malaysha, S., Suwaileh, R., Jarrar, M., Aljabari, A., Elsayed, T., Zitouni, I., 2024. ArabicNLU 2024: The First Arabic Natural Language Understanding Shared Task, in: Proceedings of ArabicNLP’2024, ACL, Bangkok, Thailand. find
  37. Khallaf, N., Arfon, E., El-Haj, M., Morris, J., Knight, D., Rayson, P., Jarrar, T.H.M., 2023. Open-source thesaurus development for under-resourced languages: a welsh case study, in: The 4th Conference on Language, Data and Knowledge (LDK2023). find
  38. Maamouri, M., Bies, A., Kulick, S., Krouna, S., Gaddeche, F., Zaghouani, W., 2010. Arabic Treebank: Part 3 V 3.2 LDC2010T08. LDC. find
  39. Malaysha, S., El-Haj, M., Ezzini, S., Khalilia, M., Jarrar, M., Nasser, S., Berrada, I., Bouamor, H., 2024a. AraFinNLP 2024: The First Arabic Financial NLP Shared Task, in: Proceedings of ArabicNLP’2024, ACL, Bangkok, Thailand. find
  40. Malaysha, S., Jarrar, M., Khalilia, M., 2023. Context-Gloss Augmentation for Improving Arabic Target Sense Verification, in: Proceedings of the 12th International Global Wordnet Conference (GWC2023), Global Wordnet Association. find
  41. Malaysha, S., Jarrar, M., Khalilia, M., 2024b. NLU-STR at SemEval-2024 Task 1: Generative-based Augmentation and Encoder-based Scoring for Semantic Textual Relatedness, in: Proceedings of the SemEval 2024 Shared Task 1 (Semantic Relatedness), ACL. find
  42. Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D., 2014. The Stanford CoreNLP Natural Language Processing Toolkit, in: Proceedings of ACL, pp. 55–60. find our card
  43. Nagoudi, E.M.B., Elmadany, A., Abdul-Mageed, M., 2022. TURJUMAN: A Public Toolkit for Neural Arabic Machine Translation . find
  44. Naser-Karajah, E., Arman, N., Jarrar, M., 2021. Current Trends and Approaches in Synonyms Extraction: Potential Adaptation to Arabic, in: Proceedings of the 2021 International Conference on Information Technology (ICIT), IEEE, Amman, Jordan. pp. 428–434. find our card
  45. Nayouf, A., Jarrar, M., zaraket, F., Hammouda, T., Kurdy, M.B., 2023. Nâbra: Syrian Arabic Dialects with Morphological Annotations, in: Proceedings of ArabicNLP, Part of the EMNLP 2023, ACL. pp. 12–23. find our card
  46. Obeid, O., Zalmout, N., Khalifa, S., Taji, D., Oudah, M., Alhafni, B., Inoue, G., Eryani, F., Erdmann, A., Habash, N., 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing, in: Proceedings of LERC, pp. 7022–7032. find our card
  47. Ousidhoum, N., Muhammad, S.H., 2024. SemRel2024: A Collection of Semantic Textual Relatedness Datasets for 14 Languages. find
  48. Ousidhoum, N., Muhammad, S.H., Abdalla, M., 2024. SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages, in: Proceedings of SemEval, ACL. find
  49. Pasha, A., Al-Badrashiny, M., Diab, M.T., El Kholy, A., Eskander, R., Habash, N., Pooleery, M., Rambow, O., Roth, R., 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic, in: LREC, pp. 1094–1101. find our card
  50. Rajpoot, P.K., Jindal, A.K., Parikh, A., 2024. Upaya at ArabicNLU Shared-Task: Arabic Lexical Disambiguation using Large Language Models, in: Proceedings of ArabicNLP, Part of ACL 2024. find
  51. Zaghouani, W., Jarrar, M., Habash, N., Bouamor, H., Zitouni, I., Diab, M., El-Beltagy, S., AbuOdeh, M., 2024. The FIGNEWS Shared Task on News Media Narratives, in: Proceedings of ArabicNLP’2024, ACL, Bangkok, Thailand. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hammouda-2024-sinatools-open-source-toolkit-arabic/