Elamur Research⌕
☰
home / papers / 2022 / A Deep CNN Architecture with Novel Pooling Layer Applied to Two Sudanese Arabic Sentiment Datasets

A Deep CNN Architecture with Novel Pooling Layer Applied to Two Sudanese Arabic Sentiment Datasets

Summary

The paper tackles sentiment classification for Sudanese Arabic. The dialect has roughly 32 million speakers, yet it has received far less attention than MSA or the Egyptian, Levantine and Moroccan varieties. Using examples, the authors show how Sudanese differs from MSA in vocabulary, verb conjugation and even word meaning. They then release two public corpora drawn from social media: SudSenti2, a positive/negative set taken from Facebook and YouTube, and SudSenti3, a three-way set taken from Twitter. Preprocessing relies on a stop-word list they built specifically for Sudanese. Their classifier, SCM, is a stacked convolutional network ending in a new pooling layer called Mean Max Average (MMA). Under cross-validation, SCM+MMA beats classical ML baselines and other neural baselines (RNN, CNN, CNN-LSTM). It also does well on two existing corpora, the Saudi SSD and the MSA hotel-review HARD. The main value of the work is the new Sudanese resources plus a controlled comparison of pooling operators. A caveat is that the paper reports somewhat different scores for the same model depending on which experiment the numbers come from.

Key points

  • New public datasets: SudSenti2 (2-class, Facebook and YouTube) and SudSenti3 (3-class, Twitter), released on GitHub along with a Sudanese stop-word list.
  • Headline accuracies from the abstract and the pooling experiment (15-fold CV): SudSenti2 92.75%, SudSenti3 84.39%, SSD 85.55%, HARD 90.01%.
  • The main comparison (10-fold CV) gives different figures: SCM+MMA scores 92.25% on SudSenti2, against RF at 87.12% and CNN-LSTM at 89.00%. On SudSenti3 it scores 85.23%, against LR at 79.37% and CNN at 83.61%.
  • In the pooling ablation, MMA beats Max, Avg and Min on all four datasets, though the margins are small. On SudSenti3, for example, MMA reaches 84.39% and Max reaches 84.11%.
  • The paper is internally inconsistent about depth: the abstract says five CNN layers, while the conclusion says four convolutional layers plus MMA.

Interesting ideas

  • I1. Ablate a single component: the network stays fixed while only the pooling operator changes (Max/Avg/Min/MMA), so any accuracy difference can be attributed to that one choice. — For Mova (learn): Mova's evaluation could swap exactly one parser or prag component at a time and report the gain next to a negative control, which keeps claims about each module honest.
  • I2. A normalization lexicon built for one specific register (Sudanese stop words) is used before modelling, instead of reusing a generic list designed for the standard language. — For Mova (tokens): crate en could load register-specific normalization tables, for example for social-media English spellings, before tagging and lemmatization, so informal input reaches the tagger in a canonical form.
  • I3. The model is trained for one variety and then checked on other varieties (Saudi dialect, MSA) to see whether it transfers. — For Mova (tree): parsers like gparse and the perceptron parser could be routinely scored on treebanks from several genres or dialects, not just the training one, to catch over-fitting to one style.
  • I4. The authors observe that the Neutral class often contains texts that mix positive and negative content, which makes three-way classification harder. — For Mova (frames): prag could represent mixed or contrastive evaluations explicitly, as separate polarity-bearing frames, instead of collapsing them into a single neutral label.
  • I5. The same model gets different scores under 10-fold and 15-fold protocols, which shows how much the evaluation setup affects the reported number. — For Mova (learn): Mova should fix and record a single evaluation protocol per benchmark, so that accuracy figures in reports are directly comparable over time.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2201.12664 · arXiv 2201.12664

@misc{mhamed2022deep,
  title = {{A Deep CNN Architecture with Novel Pooling Layer Applied to Two Sudanese Arabic Sentiment Datasets}},
  author = {Mhamed, Mustafa and Sutcliffe, Richard and Sun, Xia and Feng, Jun and Almekhlafi, Eiad and Retta, Ephrem A.},
  year = {2022},
  doi = {10.48550/arxiv.2201.12664},
  eprint = {2201.12664},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2201.12664}
}

References

46 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Abdelli, F. Guerrouf, O. Tibermacine, and B. Abdelli. Sentiment analysis of Arabic Algerian dialect using a supervised method. In 2019 International Conference on Intelligent Systems and Advanced Computing Sciences (ISACS), pages 1–6. IEEE, 2019. find
  2. N. A. Abdulla, N. A. Ahmed, M. A. Shehab, and M. Al-Ayyoub. Arabic sentiment analysis: Lexicon-based and corpus-based. In 2013 IEEE Jordan conference on applied electrical engineering and computing technologies (AEECT), pages 1–6. IEEE, 2013. find
  3. H. A. Addi and R. Ezzahir. Sampling techniques for Arabic sentiment classification: A comparative study. In Proceedings of the 3rd International Conference on Networking, Information Systems & Security, pages 1–6, 2020. find
  4. T. Al-Moslmi, M. Albared, A. Al-Shabi, N. Omar, and S. Abdullah. Arabic senti-lexicon: Constructing publicly available language resources for Arabic sentiment analysis. Journal of information science, 44(3):345–362, 2018. find
  5. M. Al Omari, M. Al-Hajj, N. Hammami, and A. Sabra. Sentiment classifier: Logistic regression for Arabic services’ reviews in Lebanon. In 2019 International Conference on Computer and Information Sciences (ICCIS), pages 1–5. IEEE, 2019. find
  6. S. Al-Saqqa, N. Obeid, and A. Awajan. Sentiment analysis for Arabic text using ensemble learning. In 2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA), pages 1–7. IEEE, 2018. find
  7. M. S. Al-shaibani, Z. Alyafeai, and I. Ahmad. Meter classification of Arabic poems using deep bidirectional recurrent neural networks. Pattern Recognition Letters, 136:1–7, 2020. find
  8. I. Al-Turaiki, M. Alshahrani, and T. Almutairi. Building predictive models for mers-cov infections using data mining techniques. Journal of Infection and Public Health, 9(6):744–748, 2016. find
  9. R. Alahmary and H. Al-Dossari. A semiautomatic annotation approach for sentiment analysis. Journal of Information Science, page 01655515211006594, 2021. find
  10. A. Alharbi, M. Taileb, and M. Kalkatawi. Deep learning in Arabic sentiment analysis: An overview. Journal of Information Science, 47(1):129–140, 2021. find
  11. A. R. Alharbi and A. Aljaedi. Predicting rogue content and Arabic spammers on twitter. Future Internet, 11(11):229, 2019. find
  12. A. Alwehaibi and K. Roy. Comparison of pre-trained word vectors for Arabic text classification using deep learning approach. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 1471–1474. IEEE, 2018. find
  13. M. Aly and A. Atiya. Labr: A large scale Arabic book reviews dataset. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 494–498, 2013. find
  14. S. N. Alyami and S. O. Olatunji. Application of support vector machine for Arabic sentiment classification using twitter-based dataset. Journal of Information & Knowledge Management, 19(01):2040018, 2020. find
  15. D. Britz. Recurrent neural networks tutorial, part 1–introduction to rnns. online] www. wildml. com, Available at: http://www.wildml. com/2015/09/recurrent-neural-networkstutorial-part-1-introduction-to-rnns/.[Accessed 02 May. 2019], 2015. ↗
  16. A. Elnagar, Y. S. Khalifa, and A. Einea. Hotel Arabic-reviews dataset construction for sentiment analysis applications. In Intelligent natural language processing: Trends and applications, pages 35–52. Springer, 2018. find
  17. F. A. Gers, J. Schmidhuber, and F. Cummins. Continual prediction using lstm with forget gates. In Neural Nets WIRN Vietri-99, pages 133–138. Springer, 1999. find
  18. M. O. Hegazi, Y. Al-Dossari, A. Al-Yahy, A. Al-Sumari, and A. Hilal. Preprocessing Arabic text on social media. Heliyon, 7(2):e06191, 2021. find
  19. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. find
  20. I. O. Hussien, K. Dashtipour, and A. Hussain. Comparison of sentiment analysis approaches using modern Arabic and Sudanese dialect. In International Conference on Brain Inspired Cognitive Systems, pages 615–624. Springer, 2018. find
  21. M. A. Jerbi, H. Achour, and E. Souissi. Sentiment analysis of code-switched Tunisian dialect: Exploring rnn-based techniques. In International Conference on Arabic Language Processing, pages 122–131. Springer, 2019. find
  22. Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel. Handwritten digit recognition with a back-propagation network. Advances in neural information processing systems, 2, 1989. find
  23. L. Lulu and A. Elnagar. Automatic Arabic dialect classification using deep learning models. procedia comput sci 142: 262–269, 2018. find
  24. S. Mdhaffar, F. Bougares, Y. Esteve, and L. Hadrich-Belguith. Sentiment analysis of Tunisian dialects: Linguistic resources and experiments. In Third Arabic Natural Language Processing Workshop (WANLP), pages 55–61, 2017. find
  25. M. Mhamed, R. Sutcliffe, X. Sun, J. Feng, E. Almekhlafi, and E. A. Retta. Improving Arabic sentiment analysis using cnn-based architectures and text preprocessing. Computational Intelligence and Neuroscience, 2021, 2021. find
  26. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013. ↗
  27. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. arXiv preprint arXiv:1310.4546, 2013. ↗
  28. A. H. Mohammad. Arabic text classification: A review. Modern Applied Science, 13(5), 2019. find
  29. A. H. Mohammad, T. Alwada’n, and O. Almomani. Arabic text categorization using support vector machine, nave bayes and neural network. In Global Science and Technology Forum Journal of Computing, volume 5, pages 108–115, 2016. find
  30. L. Moudjari, K. Akli-Astouati, and F. Benamara. An Algerian corpus and an annotation platform for opinion and emotion analysis. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 1202–1210, 2020. find
  31. H. Mulki, H. Haddad, M. Gridach, and I. Babaoglu. Empirical evaluation of leveraging named entities for Arabic sentiment analysis. arXiv preprint arXiv:1904.10195, 2019. ↗
  32. M. Nabil, M. Aly, and A. Atiya. Astd: Arabic sentiment tweets dataset. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 2515–2519, 2015. find our card
  33. O. Oueslati, E. Cambria, M. B. HajHmida, and H. Ounelli. A review of sentiment analysis research in Arabic language. Future Generation Computer Systems, 112:408–430, 2020. find our card
  34. M. Ranzato, Y.-L. Boureau, Y. LeCun, et al. Sparse feature learning for deep belief networks. Advances in neural information processing systems, 20:1185–1192, 2007. find
  35. K. Ravi and V. Ravi. A novel automatic satire and irony detection using ensembled feature selection and data mining. Knowledge-based systems, 120:15–33, 2017. find
  36. M. Rushdi-Saleh, M. T. Martín-Valdivia, L. A. Ureña-López, and J. M. Perea-Ortega. Oca: Opinion corpus for Arabic. Journal of the American Society for Information Science and Technology, 62(10):2045–2054, 2011. find
  37. M. Saad. The impact of text preprocessing and term weighting on Arabic text classification. msc. Computer Engineering Department, The Islamic University, Gaza, 2010. find
  38. R. M. Sallam, H. M. Mousa, and M. Hussein. Improving Arabic text categorization using normalization and stemming techniques. Int. J. Comput. Appl, 135(2):38–43, 2016. find
  39. K. Sayadi, M. Liwicki, R. Ingold, and M. Bui. Tunisian dialect and modern standard Arabic dataset for sentiment analysis: Tunisian election context. In Second International Conference on Arabic Computational Linguistics, ACLING, pages 35–53, 2016. find
  40. K. Smaïli. Arabic Language Processing: From Theory to Practice: 7th International Conference, ICALP 2019, Nancy, France, October 16–17, 2019, Proceedings, volume 1108. Springer Nature, 2019. find
  41. R. Socher, E. Huang, J. Pennin, C. D. Manning, and A. Ng. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. Advances in neural information processing systems, 24, 2011. find
  42. A. B. Soliman, K. Eissa, and S. R. El-Beltagy. Aravec: A set of Arabic word embedding models for use in Arabic nlp. Procedia Computer Science, 117:256–265, 2017. find
  43. Y. Tabii, M. Lazaar, M. Al Achhab, and N. Enneya. Big Data, Cloud and Applications: Third International Conference, BDCA 2018, Kenitra, Morocco, April 4–5, 2018, Revised Selected Papers, volume 872. Springer, 2018. find
  44. T. Yılmaz, A. Ergil, and B. İlgen. Deep learning-based document modeling for personality detection from turkish texts. In Proceedings of the Future Technologies Conference, pages 729–736. Springer, 2019. find
  45. O. Zaidan and C. Callison-Burch. The Arabic online commentary dataset: an annotated dataset of informal Arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 37–41, 2011. find our card
  46. Y. Zhang and B. Wallace. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. arXiv preprint arXiv:1510.03820, 2015. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mhamed-2022-deep-cnn-architecture-novel-pooling/