Elamur Research⌕
☰
home / papers / 2019 / Mazajak: An Online Arabic Sentiment Analyser

Mazajak: An Online Arabic Sentiment Analyser

Paper · 2019 · Ibrahim Abu Farha and Walid Magdy

Summary

Arabic sentiment analysis lacked a ready-to-use public tool. The only multilingual option, SentiStrength, relies on a dictionary that handles MSA but breaks down on the dialects common on social media. Mazajak fills this gap. It is an online sentiment classifier that uses a CNN whose output feeds into an LSTM. Its input is word2vec skip-gram embeddings trained on about 250M unique Arabic tweets, collected between 2013 and 2016 so that many dialects and topics are covered. Text passes through a light normalisation pipeline: unifying letter variants, collapsing elongated letters, and stripping noise. The authors evaluated the model on three tweet benchmarks: SemEval 2017 Task 4-A, ASTD and ArSAS. It slightly beats the previous best systems on the first two and scores high on ArSAS, which had no published baseline. The deployed tool is retrained on SemEval and ASTD combined and offers several access modes: single text, batch files, Twitter timelines and a Python API. Users can also send feedback that becomes new training data. The work gives Arabic NLP a reusable, dialect-tolerant sentiment component.

Key points

  • Architecture: 300-d embeddings → 1D conv (300 filters, width 3) → max pooling (size 2) → LSTM (128 cells, 20% recurrent and output dropout) → softmax, trained with Adam at lr 0.0001.
  • The embedding corpus has 250M unique tweets, far larger than the 67M tweets behind AraVec, the previous largest set.
  • Scores (F_PN, the macro-F over positive and negative only):
    • SemEval 0.63 vs 0.61 for El-Beltagy et al. (2017).
    • ASTD 0.72 vs 0.71 for the CNN/Bi-LSTM ensemble of Heikal et al. (2018).
    • ArSAS 0.90, with accuracy 0.92.
  • Data handling:
    • ArSAS was cut from over 21K to 17,784 tweets by removing the rare "mixed" class and labels with confidence of 50% or below.
    • ASTD and ArSAS were split randomly 80/20 because they have no official split.
  • The online modes are text input with a feedback form, batch files, Twitter timeline plots of sentiment over time, and a downloadable Python API. Planned extensions are Arabizi and emoji handling.

Interesting ideas

  • I1. The exact same normalisation is applied both when pretraining the embeddings and when classifying, so the vocabulary seen at inference matches the vectors learned.
  • I2. Crowd-labelled data is filtered by annotator confidence, and a sparse, ambiguous class is dropped rather than learned badly.
  • I3. The live tool lets users mark wrong outputs, and these corrections periodically flow back into retraining.
  • I4. The main metric (F_PN) averages F only over the informative polar classes, so a dominant neutral class cannot inflate the score.
  • I5. Timeline mode aggregates per-tweet labels into class counts over time for one account, turning item-level predictions into a trend view.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): run one shared normaliser for building lexicons and lemma tables and for live tokenisation, so training-time and runtime forms never drift apart.
  • I2 (learn): silver trees or frames could be kept only above an agreement threshold among witness parsers, and rare, ill-defined labels could be set aside until enough clean examples exist.
  • I3 (learn): a correction channel in the answering readers could log disputed parses or answers as candidate data for self-induced repair rules, each checked against negative controls before use.
  • I4 (frames): when evaluating irony or speech-act detection, where "none" dominates, score only the marked categories to see real detection quality.
  • I5 (world): per-sentence states such as attitude or holdings could be tracked along the event order of a story, letting readers answer questions about how a character's state changes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom255 works · 68,771 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 609 worksSemitic languages · 1,485 works

Cite

Cite as Ibrahim Abu Farha and Walid Magdy. 2019. Mazajak: An Online Arabic Sentiment Analyser. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-4621

@inproceedings{farha2019mazajak,
  title = {{Mazajak: An Online Arabic Sentiment Analyser}},
  author = {Ibrahim Abu Farha and Walid Magdy},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4621},
  url = {https://doi.org/10.18653/v1/w19-4621}
}

References

35 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohammad Al-Smadi, Omar Qawasmeh, Mahmoud Al-Ayyoub, Yaser Jararweh, and Brij Gupta. 2017b. Deep recurrent neural network vs. support vector machine for aspect-based sentiment analysis of arabic hotels’ reviews. Journal of Computational Science. ↗
  2. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16.
  3. Mohammad Al-Smadi, Bashar Talafha, Mahmoud Al-Ayyoub, and Yaser Jararweh. 2018. Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews. International Journal of Machine Learning and Cybernetics, pages 1–13. ↗
  4. Muhammad Abdul-Mageed. 2017a. Modeling arabic subjectivity and sentiment in lexical space. Information Processing & Management. ↗
  5. Muhammad Abdul-Mageed. 2017b. Not all segments are created equal: Syntactically motivated sentiment analysis in lexical space. In Proceedings of the third Arabic natural language processing workshop, pages 147–156. find
  6. Abdulaziz M. Alayba, Vasile Palade, Matthew England, and Rahat Iqbal. 2017. Arabic language sentiment analysis on health services. CoRR, abs/1702.03197. ↗
  7. Abdulaziz M. Alayba, Vasile Palade, Matthew England, and Rahat Iqbal. 2018. A combined cnn and lstm model for arabic sentiment analysis. arXiv preprint arXiv:1807.02911. ↗
  8. Muhammad Abdul-Mageed, Mona Diab, and Sandra Kübler. 2014. Samar: Subjectivity and sentiment analysis for arabic social media. Computer Speech & Language, 28(1):20–37. ↗
  9. A. Aziz Altowayan and Lixin Tao. 2016. Word embeddings for arabic sentiment analysis. In 2016 IEEE International Conference on Big Data (Big Data), pages 3820–3825. find
  10. Muhammad Abdul-Mageed, Mona T. Diab, and Mohammed Korayem. 2011. Subjectivity and sentiment analysis of modern standard arabic. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT ’11, pages 587–591, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  11. Gilbert Badaro, Ramy Baly, Hazem Hajj, Nizar Habash, and Wassim El-Hajj. 2014. A large scale arabic sentiment lexicon for arabic opinion mining. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 165–173. find
  12. Nawaf A. Abdulla, Nizar A. Ahmed, Mohammed A. Shehab, and Mahmoud Al-Ayyoub. 2013. Arabic sentiment analysis: Lexicon-based and corpus-based. In Applied Electrical Engineering and Computing Technologies (AEECT), 2013 IEEE Jordan Conference on, pages 1–6. IEEE. find
  13. Abdelghani Dahou, Shengwu Xiong, Junwei Zhou, Mohamed Houcine Haddoud, and Pengfei Duan. 2016. Word embeddings and convolutional neural network for arabic sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2418–2427. find
  14. Mahmoud Al-Ayyoub, Safa Bani Essa, and Izzat Alsmadi. 2015. Lexicon-based sentiment analysis of arabic tweets. International Journal of Social Network Mining, 2(2):101–114. ↗
  15. Kareem Darwish, Walid Magdy, and Ahmed Mourad. 2012. Language processing for arabic microblog retrieval. In Proceedings of the 21st ACM international conference on Information and knowledge management, pages 2427–2430. ACM. find
  16. Mahmoud Al-Ayyoub, Abed Allah Khamaiseh, Yaser Jararweh, and Mohammed N. Al-Kabi. 2019. A comprehensive survey of arabic sentiment analysis. Information Processing & Management, 56(2):320– 342. ↗
  17. Kareem Darwish, Walid Magdy, et al. 2014. Arabic information retrieval. Foundations and Trends R in Information Retrieval, 7(4):239–342. ↗
  18. Ahmad Al-Sallab, Hazem Hajj, Gilbert Badaro, Ramy Baly, Wassim El-Hajj, and Khaled Bashir Shaban. 2015. Deep learning models for sentiment analysis in arabic. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 9– 17. ↗
  19. Samhaa R. El-Beltagy, Mona El Kalamawy, and Abu Bakr Soliman. 2017. Niletmrg at semeval-2017 task 4: Arabic sentiment analysis. arXiv preprint arXiv:1710.08458. ↗
  20. AbdelRahim Elmadany, Hamdy Mubarak, and Walid Magdy. 2018. Arsas: An arabic speech-act and sentiment corpus of tweets. find
  21. Mohammad Al-Smadi, Omar Qawasmeh, Mahmoud Al-Ayyoub, Yaser Jararweh, and Brij Gupta. 2017a. Deep recurrent neural network vs. support vector machine for aspect-based sentiment analysis of arabic hotels’ reviews. Journal of Computational Science. ↗
  22. Mostafa Elmasry, Taysir Soliman, and Abdel-Rahman Hedar. 2014. Sentiment analysis of arabic slang comments on facebook. International Journal of Computers & Technology, 12(5):3470–3478. ↗
  23. Mike Thelwall, Kevan Buckley, Georgios Paltoglou, Di Cai, and Arvid Kappas. 2010. Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology, 61(12):2544–2558. ↗
  24. Kariman Elshakankery and Mona F. Ahmed. 2019. Hilatsa: A hybrid incremental learning approach for arabic tweets sentiment analysis. Egyptian Informatics Journal. ↗
  25. Nizar Y. Habash. 2010. Introduction to arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1):1–187. ↗
  26. Maha Heikal, Marwan Torki, and Nagwa El-Makky. 2018. Sentiment analysis of arabic tweets using deep learning. Procedia Computer Science, 142:114–122. find
  27. Mohammed Jabreel and Antonio Moreno. 2017. Sitaka at semeval-2017 task 4: Sentiment analysis in twitter based on a rich set of features. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 694–699. find
  28. Bing Liu. 2012. Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies, 5(1):1–167. ↗
  29. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  30. Mahmoud Nabil, Mohamed Aly, and Amir Atiya. 2015. Astd: Arabic sentiment tweets dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2515–2519. ↗
  31. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10, pages 79–86. Association for Computational Linguistics. ↗
  32. Arfath Pasha, Mohamed Al-Badrashiny, Mona T. Diab, Ahmed El-Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic. In LREC, volume 14, pages 1094–1101.
  33. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 502–518.
  34. Md Shoeb and Jawed Ahmed. 2017. Sentiment analysis and classification of tweets using data mining. International Research Journal of Engineering and Technology, 4(12). find
  35. Abu Bakr Soliman, Kareem Eissa, and Samhaa R. El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256 – 265. Arabic Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/farha-2019-mazajak-online-arabic-sentiment-analyser/