Elamur Research⌕
☰
home / papers / 2015 / Deep Learning Models for Sentiment Analysis in Arabic

Deep Learning Models for Sentiment Analysis in Arabic

Paper · 2015 · Ahmad Al Sallab, Hazem Hajj, Gilbert Badaro et al.

Summary

The paper addresses sentence-level sentiment classification (positive vs. negative) for Arabic. Rich morphology, ambiguity and many dialects make opinion mining in Arabic hard, and the authors present this as the first study of deep learning for the task in that language. They compare four architectures on sentences from the LDC ATB (Linguistic Data Consortium Arabic Tree Bank). Three of them receive fixed-length bag-of-words vectors built from ArSenL lexicon scores, either per lemma or summed per sentence:

  • a plain DNN;
  • a DBN with generative pre-training followed by discriminative fine-tuning;
  • a Deep Auto Encoder feeding a DBN.

The fourth, a Recursive Auto Encoder (RAE), reads raw word indices through learned 50-dimensional embeddings. It merges words pairwise into one sentence vector, choosing the merge order that lets it best reconstruct the input, and a softmax layer then classifies that vector. Among the lexicon-based models, the auto encoder compressed the sparse input most usefully. The RAE was best overall, improving average F1 by roughly 9% over earlier published results on the same data, and it needs no sentiment lexicon, which is scarce and expensive to build.

Key points

  • ArSenL gives each lemma three scores (positive, negative, neutral) that sum to 1. It covers 28,780 Arabic lemmas linked to 157,969 synsets.
  • 3,795 ArSenL entries matched words in LDC ATB, producing a sparse per-lemma vector of 11,385 dimensions (arsenl_lemma). The sentence-aggregated alternative has only 3 dimensions (arsenl_sentence).
  • The DNN was tuned on a development set, ending with 3 hidden layers of 40 tanh neurons each and a 2-neuron softmax. It used Gaussian initialization, conjugate gradient with three line searches, batches of 100 and 50 epochs.
  • The Deep Auto Encoder used layers of 100-50-20 to turn the sparse lexicon vector into a dense code before DBN classification.
  • The RAE vocabulary for LDC ATB has 31,850 words, with no stemming or stop-word removal. Test words unseen in training map to a single UNKNOWN index.

Interesting ideas

  • I1. The RAE builds a binary tree without any annotation: at each step it combines the neighbouring pair whose merged vector best regenerates both parts, so reconstruction error acts as the structure-selection criterion.
  • I2. The study pits order-blind bag-of-words inputs against an order-aware compositional model on identical data, and the compositional model wins even without the hand-built lexicon.
  • I3. Polarity is stored per lemma as a graded triple (positive/negative/neutral summing to one), not a single label, and can also be pooled to sentence level.
  • I4. The vocabulary comes strictly from training data, and all unseen test words collapse to one UNKNOWN symbol, so the evaluation honestly reflects out-of-vocabulary conditions.
  • I5. The authors suggest pre-training the embedding block with ArSenL scores, rather than random initialization, so composition starts from each word's own sentiment while still learning from context.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): the same principle could score the text → graph → text compressor, using how faithfully a meaning graph regenerates its sentence as an explainable quality signal for choosing between candidate graphs.
  • I2 (learn): keep a bag-of-words classifier as a standing negative control, so any claimed gain from tree- or frame-based reasoning must clearly beat a baseline that ignores structure.
  • I3 (world): compiled lexical knowledge could carry such graded polarity triples on lemmas, giving prag frames a soft prior when judging evaluative stance or irony.
  • I4 (tokens): tagger and lemmatizer evaluations could report results separately for tokens unseen in training, making OOV weaknesses visible instead of averaged away.
  • I5 (frames): role and stance composition in prag could be seeded with lexical priors from VerbNet or polarity lists, then corrected by sentence context, keeping both sources traceable.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Ahmad Al Sallab, Hazem Hajj, Gilbert Badaro, Ramy Baly, Wassim El Hajj and Khaled Bashir Shaban. 2015. Deep Learning Models for Sentiment Analysis in Arabic. Proceedings of the Second Workshop on Arabic Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3202

@inproceedings{sallab2015deep,
  title = {{Deep Learning Models for Sentiment Analysis in Arabic}},
  author = {Ahmad {Al Sallab}, Hazem Hajj, Gilbert Badaro, Ramy Baly, Wassim {El Hajj} and Khaled Bashir Shaban},
  booktitle = {Proceedings of the Second Workshop on Arabic Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/w15-3202},
  url = {https://doi.org/10.18653/v1/w15-3202}
}

References

24 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gilbert Badaro, Ramy Baly, Hazem Hajj, Nizar Habash, and Wassim El-Hajj. "A Large Scale Arabic Sentiment Lexicon for Arabic Opinion Mining." ANLP 2014 (2014): 165.
  2. Socher, Richard, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. "Recursive deep models for semantic compositionality over a sentiment treebank." In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1631-1642. 2013.
  3. Socher, Richard, et al. "Semi-supervised recursive autoencoders for predicting sentiment distributions." Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2011.
  4. R. Collobert and J. Weston. 2008. A unified architecture for natural language processing: deep neural networks with multitask learning. In Proceedings of ICML, pages 160–167. find
  5. Abdul-Mageed, M., Diab, M. and Korayem, M. (2011). Subjectivity and sentiment analysis of modern standard Arabic. In Proceedings of the 49th Annual Meeting of the Association for Com-putational Linguistics: Human Language Technol-ogies: short papers-Volume 2. Association for Computational Linguistics.
  6. Abdul-Mageed, M., & Diab, M. (2012). Toward building a large-scale Arabic sentiment lexicon. In Proceedings of the 6th International Global WordNet Conference (pp. 18-22). find
  7. G. E. Hinton; S. Osindero; Y. Teh, “A fast learning algorithm for deep belief nets” Neural Computation, vol. 18, pp. 1527–1554, 2006. ↗
  8. Ruslan Salakhutdinov. 2009.“Learning Deep Generative Models” PhD thesis, Graduate Department of Computer Science, University of Toronto. find
  9. Larochelle, Hugo, Yoshua Bengio, Jérôme Louradour, and Pascal Lamblin. "Exploring strategies for training deep neural networks." The Journal of Machine Learning Research 10 (2009): 1-40. ↗
  10. Bengio, Yoshua. "Practical recommendations for gradient-based training of deep architectures." In Neural Networks: Tricks of the Trade, pp. 437-478. Springer Berlin Heidelberg, 2012. find
  11. LeCun, Yann, and Yoshua Bengio. "Convolutional networks for images, speech, and time series." The handbook of brain theory and neural networks3361 (1995): 310. find
  12. Rushdi-Saleh, M., Martín-Valdivia, M. T., Ureña-López, L. A., & Perea-Ortega, J. M. "OCA: Opinion Corpus for Arabic." Journal of the American Society for Information Science and Technology 62.10, 2045-2054, 2011 ↗
  13. Rushdi-Saleh, M., Martín-Valdivia, M. T., Ureña-López, L. A., & Perea-Ortega, J. M. "Bilingual Experiments with an Arabic-English Corpus for Opinion Mining." Proceedings of Recent Advances in Natural Language Processing, 2011 find
  14. Mountassir, Asmaa, Houda Benbrahim, and Ilham Berrada. "A Cross-study of Sentiment Classification on Arabic Corpora." Research and Development in Intelligent Systems XXIX, Springer London, 2012 find
  15. Aly, M. A., & Atiya, A. F. "LABR: A Large Scale Arabic Book Reviews Dataset." Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 2013 ↗
  16. Elawady, Rasheed M., Sherif Barakat, and Nora M. Elrashidy “Sentiment Analyzer for Arabic Comments”, International Journal of Information Science and Intelligent System, 3(4): 73-86, 2014 find
  17. Al-Kabi, Mohammed N., Nawaf A. Abdulla, and Mahmoud Al-Ayyoub "An Analytical Study of Arabic Sentiments: Maktoob Case Study." Proceedings of the 8th International Conference for Internet Technology and Secured Transactions (ICITST), IEEE, 2013 find
  18. Shoukry, A., & Rafea, A. "Sentence-level Arabic Sentiment Analysis." Proceedings of International Conference on Collaboration Technologies and Systems (CTS), IEEE, 2012 find
  19. Omar, N., Albared, M., Al-Shabi, A., & Al-Moslmi, T. “Ensemble of Classification Algorithms for Subjectivity and Sentiment Analysis of Arabic Customers’ Reviews.” International Journal of Advancements in Computing Technology, 14(5), 2013 find
  20. Abbasi, Ahmed, Hsinchun Chen, and Arab Salem. "Sentiment Analysis in Multiple Languages: Feature Selection for Opinion Classification in Web forums." ACM Transactions on Information Systems (TOIS) 26.3, 2008 find
  21. Ibrahim, Hossam S., Sherif M. Abdou, and Mervat Gheith. "Sentiment Analysis for Modern Standard Arabic and Colloquial." International Journal on Natural Language Computing (IJNLC) Vol. 4, No.2, April 2015 ↗
  22. Abdul-Mageed, M., & Diab, M. T. " Subjectivity and Sentiment Analysis of Modern Standard Arabic." Proceedings of the 5th Linguistic Annotation Workshop, Association for Computational Linguistics, 2011
  23. Nabil, Mahmoud, Mohamed Aly, and Amir Atiya. “LABR: A Large Scale Arabic Sentiment Analysis Benchmark”, 2015 ↗
  24. ElSahar, H., and El-Beltagy S.R. "Building Large Arabic Multi-domain Resources for Sentiment Analysis." Computational Linguistics and Intelligent Text Processing. Springer International Publishing, 2015 find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sallab-2015-deep-learning-models-sentiment-analysis/