Elamur Research⌕
☰
home / papers / 2023 / Arabic Sentiment Analysis with Noisy Deep Explainable Model

Arabic Sentiment Analysis with Noisy Deep Explainable Model

Summary

The paper targets two weaknesses of Arabic sentiment analysis (ASA). Deep models overfit on the small amounts of labelled Arabic data, and they work as black boxes that give no reason for a positive or negative verdict. The authors build two binary classifiers, BiLSTM and CNN-BiLSTM, over padded word-index sequences. Each ends in dense and dropout layers, and a Gaussian noise layer sits just before the output layer. They test all three on two public review corpora, LABR (book reviews) and HTL (hotel reviews). They compare three variants: noise plus dropout, dropout only, and noise only. To explain individual predictions they fit LIME, a local surrogate model trained on perturbed copies of the input. Adding noise narrows the gap between training and test accuracy and raises negative-class precision. It also yields LIME word highlights that look more sensible to a human reader. The approach does not beat attention models or AraBERT on LABR, but it outperforms mBERT and several earlier CNN/LSTM baselines. The authors present it as the first application of XAI to ASA.

Key points

  • Datasets: LABR (Large Arabic Book Review) and HTL (Arabic Hotel Review). Models: BiLSTM and CNN-BiLSTM, each with a Gaussian noise layer placed directly before the final dense output.
  • Overfitting is measured as the train–test accuracy gap.
    • CNN-BiLSTM with noise and dropout: gap of 5.07% (99.85% train vs 94.78% test). With dropout only, the gap is 7.95%.
    • BiLSTM with noise and dropout: gap of 6.08%.
    • Noise only: gaps of 6.35% (BiLSTM) and 6.27% (CNN-BiLSTM).
  • For BiLSTM, negative-class precision rises from 0.56 to 0.62 when noise is added. The CNN-BiLSTM shows the same trend.
  • LABR accuracy comparison: CNN-BiLSTM reaches 88.0%. This is below Attention-BiGRU (95.6), SRU-Attention (95.1) and AraBERT (89.6). It is above Multi-chan CNN (87.5), mBERT (83.0), CNN/BiLSTM/LSTM baselines (81.6/80.7/78.5) and SVM (78.3).
  • A LIME example on a hotel review compares the two models.
    • The noise model marks "good", "very", "quiet" and "helpful" as positive evidence.
    • The dropout-only model fails to mark "good" and "helpful".

Interesting ideas

  • I1. The authors run a small grid of regularizer placements (noise+dropout, dropout only, noise only) and judge each by its train–test gap, not by test accuracy alone; this isolates what each component actually contributes. — For Mova (learn): Each new parser or tagger regularizer could get the same grid, logging the train–dev gap per variant so gains are attributed to a specific component.
  • I2. They inject Gaussian noise into the final hidden representation as a cheap regularizer for networks trained on little data. — For Mova (tree): gparse could add such a noise layer before its arc/label scoring heads when trained on small or converted treebanks, where overfitting is most likely.
  • I3. LIME perturbs the input by dropping words and fits a local linear model to see which words drive a prediction; this makes it model-agnostic. — For Mova (answer): Removing words from a story and checking whether the reader's answer changes would reveal which tokens an answer truly depends on, a useful sanity check next to the explicit frame trace.
  • I4. The paper uses explanations themselves to compare model variants: the better model is the one whose highlighted evidence matches what a human would point to. — For Mova (learn): Evaluation could check whether the frames and arguments a component relied on match gold evidence, flagging cases that are right for the wrong reason.
  • I5. Besides overall accuracy, they report precision per class, which exposed that noise mainly helped on the harder negative class. — For Mova (frames): Speech-act and irony classifiers in prag should report per-label precision, since rare labels like irony can degrade while overall accuracy stays flat.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1145/3639233.3639241 · arXiv 2309.13731

@misc{atabuzzaman2023arabic,
  title = {{Arabic Sentiment Analysis with Noisy Deep Explainable Model}},
  author = {Atabuzzaman, Md. and Shajalal, Md and Baby, Maksuda Bilkis and Boden, Alexander},
  year = {2023},
  doi = {10.1145/3639233.3639241},
  eprint = {2309.13731},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1145/3639233.3639241}
}

References

42 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Cristian Muoz Villalobos, Leonardo Mendoza Forero, Harold De Mello, Cesar Valencia, Alvaro Orjuela, Ricardo Tanscheit, and Marco Pacheco Cavalcanti. Sentimental analysis on social media comments with recurring models and pretrained word embeddings in portuguese. In Proceedings of the 2022 6th International Conference on Natural Language Processing and Information Retrieval, pages 205–209, 2022. find
  2. Bing Liu. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1):1–167, 2012. find
  3. Mahmoud Al-Ayyoub, Abed Allah Khamaiseh, Yaser Jararweh, and Mohammed N Al-Kabi. A comprehensive survey of arabic sentiment analysis. Information processing & management, 56(2):320–342, 2019. find
  4. Gilbert Badaro, Ramy Baly, Hazem Hajj, Nizar Habash, and Wassim El-Hajj. A large scale arabic sentiment lexicon for arabic opinion mining. In Proceedings of the EMNLP 2014 workshop on arabic natural language processing (ANLP), pages 165–173, 2014. find
  5. Ahmad Al Sallab, Hazem Hajj, Gilbert Badaro, Ramy Baly, Wassim El-Hajj, and Khaled Shaban. Deep learning models for sentiment analysis in arabic. In Proceedings of the second workshop on Arabic natural language processing, pages 9–17, 2015. find
  6. Ahmad Al-Sallab, Ramy Baly, Hazem Hajj, Khaled Bashir Shaban, Wassim El-Hajj, and Gilbert Badaro. Aroma: A recursive deep learning model for opinion mining in arabic as a low resource language. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):1–20, 2017. find
  7. Ramy Baly, Alaa Khaddaj, Hazem Hajj, Wassim El-Hajj, and Khaled Bashir Shaban. Arsentd-lev: A multi-topic corpus for target-based sentiment analysis in arabic levantine tweets. arXiv preprint arXiv:1906.01830, 2019. ↗ our card
  8. Abdelghani Dahou, Mohamed Abd Elaziz, Junwei Zhou, and Shengwu Xiong. Arabic sentiment classification using convolutional neural network and differential evolution algorithm. Computational intelligence and neuroscience, 2019, 2019. find
  9. Ibrahim Abu Farha and Walid Magdy. Mazajak: An online arabic sentiment analyser. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 192–198, 2019. find
  10. Wissam Antoun, Fady Baly, and Hazem Hajj. Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104, 2020. ↗ our card
  11. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016. find our card
  12. Sherin Mary Mathews. Explainable artificial intelligence applications in nlp, biomedical, and malware classification: a literature review. In Intelligent computing-proceedings of the computing conference, pages 1269–1292. Springer, 2019. find
  13. A Aziz Altowayan and Lixin Tao. Word embeddings for arabic sentiment analysis. In 2016 IEEE International Conference on Big Data (Big Data), pages 3820–3825. IEEE, 2016. find
  14. Mohamed Aly and Amir Atiya. Labr: A large scale arabic book reviews dataset. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 494–498, 2013. find
  15. Hady ElSahar and Samhaa R El-Beltagy. Building large arabic multi-domain resources for sentiment analysis. In International conference on intelligent text processing and computational linguistics, pages 23–34. Springer, 2015. find
  16. Noura Farra, Elie Challita, Rawad Abou Assi, and Hazem Hajj. Sentence-level and document-level sentiment mining for arabic texts. In 2010 IEEE international conference on data mining workshops, pages 1114–1119. IEEE, 2010. find
  17. Muhammad Abdul-Mageed, Mona Diab, and Mohammed Korayem. Subjectivity and sentiment analysis of modern standard arabic. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 587–591, 2011. find
  18. Ibrahim Abu Farha and Walid Magdy. A comparative study of effective approaches for arabic sentiment analysis. Information Processing & Management, 58(2):102438, 2021. find
  19. Amira Shoukry and Ahmed Rafea. Sentence-level arabic sentiment analysis. In 2012 international conference on collaboration technologies and systems (CTS), pages 546–550. IEEE, 2012. find
  20. Amira Shoukry and Ahmed Rafea. A hybrid approach for sentiment classification of egyptian dialect tweets. In 2015 First International Conference on Arabic Computational Linguistics (ACLing), pages 78–85. IEEE, 2015. find
  21. Rehab M Duwairi, Raed Marji, Narmeen Sha’ban, and Sally Rushaidat. Sentiment analysis in arabic tweets. In 2014 5th international conference on information and communication systems (ICICS), pages 1–6. IEEE, 2014. find
  22. Hamada Nayel, Eslam Amer, Aya Allam, and Hanya Abdallah. Machine learning-based model for sentiment and sarcasm detection. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 386–389, 2021. find our card
  23. Sarah Alhumoud, Tarfa Albuhairi, and Wejdan Alohaideb. Hybrid sentiment analyser for arabic tweets using r. In 2015 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K), volume 1, pages 417–424. IEEE, 2015. find
  24. Oumaima Oueslati, Erik Cambria, Moez Ben HajHmida, and Habib Ounelli. A review of sentiment analysis research in arabic language. Future Generation Computer Systems, 112: 408–430, 2020. find our card
  25. Maha Heikal, Marwan Torki, and Nagwa El-Makky. Sentiment analysis of arabic tweets using deep learning. Procedia Computer Science, 142:114–122, 2018. find
  26. Nizar Habash, Houda Bouamor, Hazem Hajj, Walid Magdy, Wajdi Zaghouani, Fethi Bougares, Nadi Tomeh, Ibrahim Abu Farha, and Samia Touileb. Proceedings of the sixth arabic natural language processing workshop. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, 2021. find
  27. Amey Hengle, Atharva Kshirsagar, Shaily Desai, and Manisha Marathe. Combining context-free and contextualized representations for arabic sarcasm detection and sentiment identification. arXiv preprint arXiv:2103.05683, 2021. ↗ our card
  28. Mohamed A Zahran, Ahmed Magooda, Ashraf Y Mahgoub, Hazem Raafat, Mohsen Rashwan, and Amir Atyia. Word representations in vector space and their applications for arabic. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 430–443. Springer, 2015. find
  29. Md Atabuzzaman, Md Shajalal, M Elius Ahmed, Masud Ibn Afjal, and Masaki Aono. Leveraging grammatical roles for measuring semantic similarity between texts. IEEE Access, 9:62972–62983, 2021. find
  30. Md Shajalal, Md Zia Ullah, Abu Nowshed Chy, and Masaki Aono. Query subtopic diversification based on cluster ranking and semantic features. In 2016 International Conference On Advanced Informatics: Concepts, Theory And Application (ICAICTA), pages 1–6. IEEE, 2016. find
  31. Md Shajalal and Masaki Aono. Coverage-based query subtopic diversification leveraging semantic relevance. Knowledge and Information Systems, 62:2873–2891, 2020. find
  32. Obeida ElJundi, Wissam Antoun, Nour El Droubi, Hazem Hajj, Wassim El-Hajj, and Khaled Shaban. hulmona: The universal language model in arabic. In Proceedings of the fourth arabic natural language processing workshop, pages 68–77, 2019. find
  33. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. Bert-cnn for offensive speech identification in social media. In Proceedings of the Fourteenth Workshop on Semantic Evaluation", Barcelona (online)", International Committee for Computational Linguistics, 2020. find
  34. Anshul Wadhawan. Arabert and farasa segmentation based approach for sarcasm and sentiment detection in arabic tweets. arXiv preprint arXiv:2103.01679, 2021. ↗ our card
  35. Fatemah Husain and Ozlem Uzuner. Leveraging offensive language for sarcasm and sentiment detection in arabic. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 364–369, 2021. find our card
  36. Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. A survey of the state of explainable ai for natural language processing. arXiv preprint arXiv:2010.00711, 2020. ↗ our card
  37. Zhonghui You, Jinmian Ye, Kunming Li, Zenglin Xu, and Ping Wang. Adversarial noise layer: Regularize neural network by adding noise. In 2019 IEEE International Conference on Image Processing (ICIP), pages 909–913. IEEE, 2019. find
  38. Arvind Neelakantan, Luke Vilnis, Quoc V Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, and James Martens. Adding gradient noise improves learning for very deep networks. arXiv preprint arXiv:1511.06807, 2015. ↗
  39. Mohamed Berrimi, Mourad Oussalah, Abdelouahab Moussaoui, and Mohamed Saidi. Attention mechanism architecture for arabic sentiment analysis. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(4):1–26, 2023. find
  40. Saja Al-Dabet and Sara Tedmori. Sentiment analysis for arabic language using attention-based simple recurrent unit. In 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS), pages 1–6. IEEE, 2019. find
  41. Abdelghani Dahou, Shengwu Xiong, Junwei Zhou, and Mohamed Abd Elaziz. Multi-channel embedding convolutional neural network model for arabic sentiment classification. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 18(4):1–23, 2019. find
  42. Hanane Elfaik et al. Deep bidirectional lstm network learning-based sentiment analysis for arabic text. Journal of Intelligent Systems, 30(1):395–412, 2021. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/atabuzzaman-2023-arabic-sentiment-analysis-noisy-deep/