Elamur Research⌕
☰
home / papers / 2026 / Alkhalil Corpus: An Open-Source Thematic and Lemmatized Corpus for Modern Standard Arabic

Alkhalil Corpus: An Open-Source Thematic and Lemmatized Corpus for Modern Standard Arabic

Paper · 2026 · Samir Belayachi and Azzeddine Mazroui

Summary

Arabic lacks large, openly licensed annotated corpora. Its rich morphology, the usual absence of short-vowel diacritics, and competing spelling variants make annotation expensive. The authors release two Modern Standard Arabic resources on Hugging Face.

The first is a roughly 76-million-word collection with a topic label per document. Its texts, all published after 1850, span ten themes and originate in three regions: the Gulf, the Middle East and North Africa's Maghreb. Sources include news portals and two digital libraries. The raw web text was converted to UTF-8, stripped of markup and non-Arabic segments, and deduplicated by comparing TF-IDF vectors with cosine similarity.

The second is a sub-corpus of about one million words with lemma labels. Two independent lemmatizers, Alkhalil and MADAMIRA, were run over it. Where they agreed, their lemma was accepted automatically. Linguists hand-resolved only the conflicting words, and only in a selected set of longer sentences.

The outcome is a large thematic corpus plus a lemma-annotated corpus built cheaply but checked by humans. Both are useful for training and evaluating Arabic NLP tools.

Key points

  • The thematic corpus holds 76,259,036 words in 169,918 documents across ten themes. Sport, Policy, Economics and Science contribute the most documents, while Art, Sociology and Literature have very few.
  • Sources are news sites (Al Jazeera, BBC Arabic, CNN Arabic, Hespress, Al Yaoum24, Al Bayan, etc.) and the Hindawi and Ashamela digital libraries.
  • Near-duplicates were removed by pairwise TF-IDF cosine similarity. Any pair scoring above 0.75 lost one of its documents.
  • On the lemma sub-corpus, Alkhalil and MADAMIRA:
    • gave the same lemma for 76.86% of words;
    • gave different lemmas for 18.97%;
    • left 4.17% unanalysed by at least one tool.
    • Coverage was 97.18% for Alkhalil and 97.05% for MADAMIRA.
  • Fully agreeing sentences longer than four words, totalling 898,180 words, were accepted without review. From 5,449 longer sentences (about 100,000 words), roughly 15,000 conflicting words were resolved by two experts on a collaborative online platform.

Interesting ideas

  • I1. Two unrelated analyzers act as mutual witnesses: tokens where they agree become silver labels, and human effort goes only to their disagreements, which shrinks manual work to a small slice of the data.
  • I2. Acceptance is decided per sentence, not per token: a whole sentence counts as trusted only when every word agrees, which avoids keeping half-reliable sentences.
  • I3. Agreement between the tools was measured against sentence length and drops as sentences grow, and that curve then decided which sentences got automatic versus manual treatment.
  • I4. A cheap TF-IDF plus cosine near-duplicate filter with a fixed threshold keeps reposted articles from skewing word statistics.
  • I5. Lemmas are tied to dictionary entries and supplied by a morphological analyzer, then reconciled across tools, which yields a consistent canonical form for downstream tasks.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Silver UD data could be built the same way, accepting arcs or labels where the witness parsers coincide and routing only conflicts to the LLM teacher tier or a human.
  • I2 (learn): Treebank silver data could be gated on full-sentence witness agreement, so training never mixes confident and doubtful arcs within one tree.
  • I3 (learn): Plotting witness-parser agreement by sentence length could set data-driven thresholds for when silver trees are trusted and when they go to repair or review.
  • I4 (world): Deduplicating source texts before compiling lexical frequencies or knowledge-graph facts would stop syndicated stories from inflating counts or repeating the same claims.
  • I5 (tokens): The lemmatizer in crate en could be cross-checked against a second independent lemmatizer, with disagreements logged as an explainable list of hard cases for rule fixes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Mohamed I UniversityOujda · Morocco4 works · 433 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

💡 Concepts 1Key ideas and methods explained on our concept pages.Lemmatization · 20 works

Cite

Cite as Samir Belayachi and Azzeddine Mazroui. 2026. Alkhalil Corpus: An Open-Source Thematic and Lemmatized Corpus for Modern Standard Arabic. Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.abjadnlp-1.27

@inproceedings{belayachi2026alkhalil,
  title = {{Alkhalil Corpus: An Open-Source Thematic and Lemmatized Corpus for Modern Standard Arabic}},
  author = {Samir Belayachi and Azzeddine Mazroui},
  booktitle = {Proceedings of the 2nd Workshop on NLP for Languages Using Arabic Script},
  year = {2026},
  doi = {10.18653/v1/2026.abjadnlp-1.27},
  url = {https://doi.org/10.18653/v1/2026.abjadnlp-1.27}
}

References

13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Yonatan Belinkov, Nizar Habash, Adam Kilgarriff, Noam Ordan, Ryan Roth, Vıt Suchomel, and 1 others. 2013. artenten: a new, vast corpus for arabic. Proceedings of WACL, 20. find
  2. M. Boudchiche and A. Mazroui. 2015. Enrichment of the nemlar corpus with the lemma label. In Study Day “Arabic Language Resources for NLP: Construction, Standardization, Management and Operation”, Rabat, Morocco. November 26, 2015. find
  3. Mohamed Boudchiche, Azzeddine Mazroui, Mohamed Ould Abdallahi Ould Bebah, Abdelhak Lakhouaja, and Abderrahim Boudlal. 2017. Alkhalil morpho sys 2: A robust arabic morpho-syntactic analyzer. Journal of King Saud University-Computer and Information Sciences, 29(2):141–146. ↗
  4. Imad Zeroual, Dirk Goldhahn, Thomas Eckart, and Abdelhak Lakhouaja. 2019. Osian: Open source international arabic news corpus-preparation and integration into the clarin-infrastructure. In Proceedings of the fourth arabic natural language processing workshop, pages 175–182. ↗
  5. Imad Zeroual and Abdelhak Lakhouaja. 2018. Data science in light of natural language processing: An overview. Procedia Computer Science, 127:82–91. find
  6. Taha Zerrouki and Amar Balla. 2017. Tashkeela: Novel corpus of arabic vocalized texts, data for auto-diacritization systems. Data in brief, 11:147. find
  7. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
  8. Christopher Manning. 1999. Foundations of statistical natural language processing. The MIT Press. find
  9. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK. ↗
  10. Naoual Nassiri, Abdelhak Lakhouaja, and Violetta Cavalli-Sforza. 2018. Modern standard arabic readability prediction. In ARABIC LANGUAGE PRO-CESSING: FROM THEORY TO PRACTICE, volume 782 of Communications in Computer and Information Science, pages 120–133. Sidi Mohammed Ben Abdellah Univ, Natl Sch Appl Sci; Arabic Language Engn Soc Morocco; Ctr Natl Rech Sci Tech; Acad Hassan II Sci Tech; Ecole Natl Sci Appliquees Fes; LISA; Fac Sci Tech; Fac Sci Scharia. 6th International Conference on Arabic Language Processing (ICALP), Fez, MOROCCO, OCT 11-12, 2017. find
  11. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland. European Language Resources Association (ELRA).
  12. Ferial Senator, Abdelaziz Lakhfif, Imene Zenbout, Hanane Boutouta, and Chahrazed Mediani. 2025. Leveraging chatgpt for enhancing arabic nlp: Application for semantic role labeling and cross-lingual annotation projection. IEEE Access. ↗
  13. Ibtissam Touahri and Azzeddine Mazroui. 2021. Studying the effect of characteristic vector alteration on arabic sentiment classification. Journal of King Saud University-Computer and Information Sciences, 33(7):890–898. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/belayachi-2026-alkhalil-corpus-open-source-thematic/