Elamur Research⌕
☰
home / papers / 2019 / Automatic Text Tagging of Arabic News Articles Using Ensemble Deep Learning Models

Automatic Text Tagging of Arabic News Articles Using Ensemble Deep Learning Models

Paper · 2019 · Ashraf Elnagar, Omar Einea and Ridhwan Al-Debsi

Summary

The paper tackles the shortage of large, freely available data for assigning a single topic label to Arabic documents. The authors built SANAD, a corpus of Modern Standard Arabic news articles scraped from alarabiya.net, alkhaleej.ae and akhbarona.com, and released it on Mendeley. Across the three subsets, as many as seven topics appear, namely Sports, Politics, Religion, Finance, Culture, Tech and Medical. Cleaning was minimal, limited to stripping Latin characters and punctuation, with no stemming or feature engineering. On this data they trained nine neural classifiers: CNN, GRU, LSTM, BiGRU, BiLSTM, attention-based HANGRU and HANLSTM, and the hybrids CGRU and CLSTM. Each model was trained on one subset and tested on all three. Overall accuracy ranged from 93.43% (CGRU) to 95.81% (HANGRU). Accuracy dropped sharply when training and test sources differed. Majority-vote ensembles added small further gains. The work matters because it provides an open benchmark of about 200k articles and shows that deep models can replace heavy Arabic-specific preprocessing.

Key points

  • SANAD splits: Arabiya has 5 categories (22,203 train / 4,075 test), Khaleej 7 (42,000 / 3,500) and Akhbarona 7 (42,000 / 4,900). Khaleej is balanced; the other two are not.
  • Source-specific curation shaped each subset:
    • Khaleej: the portal's tags were vague, so articles were labelled by hand.
    • Arabiya: "Iran News" was folded into Politics and the thin Culture class was removed.
    • Akhbarona: the undersized Religion class was topped up from Alanba.com.
  • On matching test sets, the best results were HANGRU at 96.94% on KH-7 and 94.63% on AB-7, and CNN at 96.05% on AR-5.
  • Cross-source testing exposed large drops. Models trained on Khaleej scored 75.04–87.12% on Arabiya and 66.38–76.40% on Akhbarona.
  • A greedy search over model combinations produced up to 459 majority-vote ensembles in some experiments. These beat the best single model in 8 of 9 experiments, gaining at most 2.1%. HANGRU appeared in the top ensembles 7 times, CNN 6 times and BiGRU only once.

Interesting ideas

  • I1. Full source-by-source train/test matrix: training on each corpus and testing on every other one shows how much a model depends on one outlet's style, a gap that in-domain scores hide.
  • I2. Counting each model's share in winning ensembles: tallying how often a model appears in the best combinations measures its complementary value, not just its solo accuracy.
  • I3. Expecting small ensemble gains when single models are already strong: voting helped least where individual accuracy was highest, which tempers expectations about stacking more voters.
  • I4. Repairing label sets at collection time: merging a niche class into its parent, dropping a starved one, and filling a thin class from a similar outlet keeps classes usable without inventing labels.
  • I5. Robustness as "above average across experiments": a model was judged by how often it beat the mean across all nine runs, as with BiLSTM in eight of them, rather than by one peak score.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): evaluating perceptron and gparse parsers trained on one treebank against other genres would show where silver data from a single source misleads training.
  • I2 (learn): the same tally over witness-parser agreement sets could reveal which witnesses add independent evidence and which only repeat others.
  • I3 (learn): it suggests spending extra witness parsers mainly on hard, low-agreement sentences rather than on all silver data.
  • I4 (world): ontology categories compiled from WordNet or Wikidata could get the same treatment, folding sparse classes upward instead of keeping near-empty nodes.
  • I5 (tree): choosing between parser variants by consistency across treebanks and dialect conversions would favour parsers that rarely collapse over ones that occasionally excel.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Ashraf Elnagar, Omar Einea and Ridhwan Al-Debsi. 2019. Automatic Text Tagging of Arabic News Articles Using Ensemble Deep Learning Models. Proceedings of the 3rd International Conference on Natural Language and Speech Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{elnagar2019automatic,
  title = {{Automatic Text Tagging of Arabic News Articles Using Ensemble Deep Learning Models}},
  author = {Ashraf Elnagar, Omar Einea and Ridhwan Al-Debsi},
  booktitle = {Proceedings of the 3rd International Conference on Natural Language and Speech Processing},
  year = {2019},
  url = {https://aclanthology.org/W19-7409/}
}

References

31 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. El-Halees A. 2007. Arabic text classification using maximum entropy. The Islamic University Journal (Series of Natural Studies and Engineering), 15:167–167. find
  2. Charu C. Aggarwal and Cheng Xiang Zhai. 2012. Mining Text Data. Springer Publishing Company, Incorporated. ↗
  3. Mohammed N. Al-Kabi, Qasem A. Al-Radaideh, and Khalid W. Akkawi. 2011. Benchmarking and assessing the performance of arabic stemmers. Journal of Information Science, 37(2):111–119. ↗
  4. Rehab Duwairi, Mohammad Nayef Al-Refai, and Natheer Khasawneh. 2009. Feature reduction techniques for arabic text categorization. Journal of the American Society for Information Science and Technology, 60(11):2347–2352. ↗
  5. Omar Einea, Ashraf Elnagar, and Ridhwan Al Debsi. 2019. Sanad: Single-label arabic news articles dataset for automatic text categorization. Data in Brief, page 104076. ↗
  6. Mohamed El Kourdi, Amine Bensaid, and Tajje-eddine Rachidi. 2004. Automatic arabic document categorization based on the naÏve bayes algorithm. In Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, Semitic ’04, pages 51–58, Stroudsburg, PA, USA. Association for Computational Linguistics.
  7. Abdelkader El Mahdaouy, Eric Gaussier, and Saı̈d Ouatik El Alaoui. 2017. Arabic text classification based on word and document embeddings. In Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2016, pages 32–41, Cham. Springer International Publishing. find
  8. T.M. Eldos. 2003. Arabic text data mining: a root-based hierarchical indexing model. International Journal of Modelling and Simulation, 23(3):158– 166. ↗
  9. A. Elnagar and O. Einea. 2016. Brad 1.0: Book reviews in arabic dataset. In 2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA), pages 1–8. find
  10. Ashraf Elnagar, Yasmin S. Khalifa, and Anas Einea. 2018a. Hotel Arabic-Reviews Dataset Construction for Sentiment Analysis Applications, pages 35–52. Springer International Publishing. find
  11. Ashraf Elnagar, Leena Lulu, and Omar Einea. 2018b. An annotated huge dataset for standard and colloquial arabic reviews for subjective sentiment analysis. Procedia Computer Science, 142:182 – 189. Arabic Computational Linguistics. find
  12. Wael Hadi, S Al Hawari, Fadi Thabtah, and Jafar Ababneh. 2008. Naive bayesian and k-nearest neighbour to categorize arabic text data. In The 22nd annual European Simulation and Modelling Conference (ESM’2008), pages 196–200. find
  13. F. Harrag, E. El-Qawasmeh, and P. Pichappan. 2009. Improving arabic text categorization using decision trees. In 2009 First International Conference on Networked Digital Technologies, pages 110–115. find
  14. Colin Raffel and Daniel PW Ellis. 2015. Feed-forward networks with attention can solve some long-term memory problems. arXiv preprint arXiv:1512.08756. ↗
  15. Fouzi Harrag, Eyas El-Qawasmah, and Abdul Malik S. Al-Salman. 2011. Stemming as a feature reduction technique for arabic text categorization. 2011 10th International Symposium on Programming and Systems, pages 128–133. find
  16. Motaz Saad. 2010. The Impact of Text Preprocessing and Term Weighting on Arabic Text Classification. Master’s thesis, Computer Engineering Dept., Islamic University of Gaza, Palestine. ↗
  17. Ismail Hmeidi, Mahmoud Al-Ayyoub, Nawaf Abdulla, Abdalrahman Almodawar, Raddad Abooraig, and Nizar A. Ahmed. 2014. Automatic arabic text categorization: A comprehensive comparative study. Journal of Information Science, 41:114–124. ↗
  18. Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, Jürgen Schmidhuber, et al. 2001. Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. find
  19. Thorsten Joachims. 1998. Text categorization with support vector machines: Learning with many relevant features. In Machine Learning: ECML-98, pages 137–142, Berlin, Heidelberg. Springer Berlin Heidelberg. ↗
  20. Thorsten Joachims. 2002. Learning to Classify Text Using Support Vector Machines: Methods, Theory and Algorithms. Kluwer Academic Publishers, Norwell, MA, USA. find
  21. Mohammad Khorsheed and Abdulmohsen Al-thubaity. 2013. Comparative evaluation of text classification techniques using a large diverse arabic dataset. Language Resources and Evaluation, 47:513–538. ↗
  22. SVandana Korde and C. Namrata Mahender. 2012. Text classification and classifiers: A survey. IJAIA Journal, 3(2):85–99. ↗
  23. Yuancheng Li, Xiangqian Nie, and Rong Huang. 2018. Web spam classification method based on deep belief networks. Expert Systems with Applications, 96:261 – 270. ↗
  24. Leena Lulu and Ashraf Elnagar. 2018. Automatic arabic dialect classification using deep learning models. Procedia Computer Science, 142:262 – 269. Arabic Computational Linguistics. find
  25. Abdelwadood Moh’d Mesleh. 2011. Feature sub-set selection metrics for arabic text classification. Pattern Recogn. Lett., 32(14):1922–1929. ↗
  26. Abdelwadood Moh’d A Mesleh. 2007. Chi square feature extraction based svms arabic language text categorization system. Journal of Computer Science, 3(6):430–435. Exported from https://app.dimensions.ai on 2019/02/03. ↗
  27. Aytuğ Onan. 2018. An ensemble scheme based on language function analysis and feature engineering for text genre classification. Journal of Information Science, 44(1):28–47. find
  28. Hassan Sawaf, Jörg Zaplo, and Hermann Ney. 2001. Statistical classification methods for arabic news articles. In Arabic Natural Language Processing in ACL2001, pages 1–6. find
  29. Fabrizio Sebastiani. 2002. Machine learning in automated text categorization. ACM Comput. Surv., 34(1):1–47. ↗
  30. Muhammad Fahad Umer and M. Sikander Hayat Khiyal. 2007. Classification of textual documents using learning vector quantization. Information Technology Journal, 6:154–159. ↗
  31. Qussai Yaseen and Ismail Hmeidi. 2014. Extracting the roots of arabic words without removing affixes. Journal of Information Science, 40(3):376–385. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/elnagar-2019-automatic-text-tagging-arabic-news/