Elamur Research⌕
☰
home / papers / 2004 / Automatic Arabic Document Categorization Based on the Naïve Bayes Algorithm

Automatic Arabic Document Categorization Based on the Naïve Bayes Algorithm

Paper · 2004 · Mohamed El Kourdi, Amine Bensaid and Tajje-eddine Rachidi

Summary

The paper tests whether a simple statistical classifier can sort unvowelled Arabic web pages into topic directories, the kind of browsing structure portals and search engines offer. The authors collected 1,500 articles from aljazeera.net, 300 in each of five topics: Health, Business, Culture, Science and Sport. Each document is cleaned by dropping stopwords and stripping any vowel marks. Every remaining word is then reduced to its root with the El Kourdi (2004) root extractor, which copes with broken plurals and hollow verbs. A Naïve Bayes classifier with smoothed word probabilities is trained on these roots, and TF-IDF picks the most useful 50 to 2,000 terms. With leave-one-out cross validation and 2,000 roots, average accuracy reaches 68.78%. A small, hand-picked evaluation set scores 62%. These numbers are close to an earlier study (Yahyaoui, 2001) that used a different stemmer and different data. The authors take this as a sign that NB on Arabic is not very sensitive to the choice of root extractor. They name EM with unlabeled data, SVM and AdaBoost as next steps.

Key points

  • Data: 5 categories × 300 documents from aljazeera.net (1,500 total), plus a separate evaluation set of 10 manually chosen documents per category.
  • The root extractor (El Kourdi, 2004) is reported to find the correct root in over 97% of words in web text. Word probabilities use m-estimate smoothing, with m equal to the vocabulary size.
  • Cross validation used four train/test splits (1/3–2/3, 1/2–1/2, 2/3–1/3, leave-one-out) and term counts from 50 to 5,000:
    • Error falls steadily as the training share grows, from about 66–75% with the smallest training split to about 31–37% with leave-one-out.
    • Leave-one-out with 2,000 roots gives 68.78% average accuracy, and the best single category reaches 92.8%.
  • On the evaluation set, overall accuracy is 62%. Per category: Sport 90%, Business 70%, Science 60%, Health 50%, Culture 40%.
  • Comparison with Yahyaoui (2001), which used 12 categories × 100 documents: 75.6% in cross validation and 50% on its evaluation set, versus 68.78% and 62% here.

Interesting ideas

  • I1. Mapping every derived word form to one shared canonical root shrinks the index and pools evidence from related words, which matters most for morphologically rich languages.
  • I2. Repeating the same experiment while shrinking the training share (from one third up to leave-one-out) shows directly how much accuracy depends on data volume.
  • I3. Besides cross validation, the authors test on a small, separately collected set chosen to cover each category's variability, which exposes a gap between the 68.78% and 62% scores.
  • I4. Reporting accuracy per category reveals very uneven behaviour (Sport at 90%, Culture at 40%) that a single average would conceal.
  • I5. A smoothed bag-of-roots Naïve Bayes model is cheap to build and gives a meaningful accuracy floor for topic decisions.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Optionally grouping lemmas into derivational families (for example, "decide/decision") could help the lemmatizer feed downstream frame matching without bloating the vocabulary.
  • I2 (learn): Learning curves over silver-data fractions would show whether a parser or reader component is data-starved or has plateaued.
  • I3 (learn): A hand-curated, diversity-oriented held-out set for each skill would catch overfitting to the training distribution that random splits hide.
  • I4 (answer): Breaking test scores down by question type or text genre would show which readers need work instead of hiding weak areas behind one number.
  • I5 (learn): It could serve as a deliberately structure-blind negative-control baseline, showing that Mova's tree- and frame-based answers beat what word counts alone can achieve.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Al Akhawayn UniversityIfrane · Morocco3 works · 247 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksInformation Systems · 1,437 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Mohamed El Kourdi, Amine Bensaid and Tajje-eddine Rachidi. 2004. Automatic Arabic Document Categorization Based on the Naïve Bayes Algorithm. Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kourdi2004automatic,
  title = {{Automatic Arabic Document Categorization Based on the Naïve Bayes Algorithm}},
  author = {Mohamed {El Kourdi}, Amine Bensaid and Tajje-eddine Rachidi},
  booktitle = {Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages},
  year = {2004},
  doi = {10.3115/1621804.1621819},
  url = {https://doi.org/10.3115/1621804.1621819}
}

References

24 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. R. Al-Shalabi, and M. Evens, "A computational morphology system for Arabic,” In Workshop on Computational Approaches to Semitic Languages, COLING-ACL98, 1998.
  2. B. Cestink, "Estimating probabilities: A crucial task in machine learning," Proceedings of the Ninth European Conference on Artificial Intelligence, pp. 147–149, London, 1990. find
  3. K. Crammer and Y. Singer, “A Family of Additive Online Algorithms for Category Ranking,” JMLR, v. 3, pp. 1025-1058, Feb. 2003. find
  4. R. H. Creecy, B. M. Masand, S. J. Smith, and D. L. Waltz, “Trading mips and memory for knowledge engineering,” Communication of the ACM, Vol. 35, No. 8, pp. 48–64, August 1992. ↗
  5. M. El Kourdi, T. Rachidi, and A. Bensaid, "A concatenative approach to Arabic word root extraction," in progress, 2004. find
  6. Y.C. Fang, S. Parthasarathy and F. Schwartz, "Using clustering to boost text classification," ICDM Workshop on Text Mining (TextDM'01), 2001. find
  7. Y. Houmame, Towards an Arabic Information Retrieval System, MS thesis, AlAkhawayn University, Morocco, 1999. find
  8. T. Joachims, Learning to classify text using SVM, Kluwer Academic Publishers, 2002. find
  9. K. Lang, "Newsweeder: Learning to filter netnews," Proceedings of the Twelfth International Conference on Machine Learning, 1995. ↗
  10. D. Lewis, M. Ringnette, "Comparison of two learning algorithms for text categorization," Proceedings of the Third Annual Symposium on Document Analysis and Information Retrieval (SDAIR'94), 1994. find
  11. D. Lewis, Yiming Yang, Tony G. Rose, Fan Li, “A New Benchmark Collection for Text Categorization Research,” JMLR, v. 5, pp. 361-397, Apr. 2004. find
  12. T. Mitchell. Machine learning. McGraw Hill, 1997. find
  13. K. Nigam, A. K. McCallum, S. Thrun, and T.Mitchell, "Text classification from labeled and unlabeled documents using EM," Machine Learning, vol. 39, pp. 103–134, 2000. ↗
  14. T. Rachidi, O. Iraqi, M. Bouzoubaa, A. Ben Al Khattab, M. El Kourdi, A. Zahi, and A. Bensaid, “Barq: distributed multilingual Internet search engine with focus on Arabic language,” Proceedings of IEEE Conf. on Sys., Man and Cyber., Washington DC, October 5-8, pp. , 2003. find
  15. M. Rogati and Y. Yang. “High-performing feature selection for text classification,” ACM CIKM 2002. find
  16. Sakhr software company's www.sakhrsoft.com, 2004. website: G. Salton and C. S. Yang, "On the specification of term values in automatic indexing", Journal of Documentation, Vol. 29, No. 4, pp. 351–372, 1973. find
  17. F. Sebastiani, “Machine learning in automated text categorization,” ACM Computing Surveys, v.34 n.1, p.1-47, March 2002. ↗
  18. K. Tzeras and S. Hartman, "Automatic indexing based on Bayesian inference networks," Proc 16 th Ann Int ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'93), pp. 22–34, 1993. find
  19. (Wiene and Pedersen, 1995) E. Wiener, J. O. Pedersen, and A. S. Zeigend, "A neural network approach to topic spotting," Proceedings of the Fourth Symposium on Document Analysis and Information Retrieval (SDAIR'95), 1995. find
  20. M. Yahyaoui, "Toward an Arabic web page classifier," Master project. AUI. 2001. find
  21. Y. Yang, “An evaluation of statistical approaches to text categorization,” Journal of Information Retrieval, Vol. 1, Number 1-2, pp. 69–90, 1999. ↗
  22. Y. Yang and X. Liu, “A re-examination of text categorization methods,” Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'99), pp 42–49, 1999. find
  23. R. B. Yates, and B. R. Neto, Modern information retrieval. Addison-Wesley ISBN 0-201-39829-X, 1999. find
  24. Yang, Y., Pedersen J.P. A Comparative Study on Feature Selection in Text Categorization Proceedings of the 14th International Conference on Machine Learning, pp. 412-420, 1997. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kourdi-2004-automatic-arabic-document-categorization-based/