Elamur Research⌕
☰
home / papers / 2021 / Arabic aspect sentiment polarity classification using BERT

Arabic aspect sentiment polarity classification using BERT

Summary

The paper addresses aspect sentiment polarity classification for Arabic. Given a review and one of its aspects or targets, the system decides whether the opinion about that aspect is positive, negative or neutral. This corresponds to Task 3 of the SemEval-2016 ABSA setup. Earlier Arabic systems mostly fed static vectors such as word2vec or fastText into CNN, LSTM, attention or memory networks, and often added hand-crafted features and lexicons. The authors instead fine-tune a pre-trained BERT encoder and treat the problem as sentence-pair classification. The review goes in as the first segment, the aspect terms form an auxiliary second segment, and the [CLS] representation is passed to a single linear layer. No external resources or feature engineering are involved. They test on three public Arabic datasets: hotel reviews, the human-annotated book review set HAAD, and a news corpus. On all three, this deliberately plain baseline beats previously reported state-of-the-art results. The work shows that a contextual encoder with a well-chosen input format can replace heavier architectures and lexicons for a morphologically rich, less-resourced language.

Key points

  • The model reaches 89.51% accuracy for hotel reviews in Arabic, 73.23% for HAAD and 85.73% for Arabic news, surpassing the previous best on every dataset.
  • Two input variants are compared:
    • BERT-single uses the review alone.
    • BERT-pair adds the aspect as an auxiliary sentence, and the case study shows it handles sentences whose aspects carry conflicting polarities (e.g. bad service, beautiful hotel).
  • Keeping BERT's weights frozen gave clearly inadequate results, so task-specific fine-tuning was essential.
  • The datasets are public on GitHub: msmadi/HAAD, msmadi/ABSA-Hotels and malayyoub/ABSA-for-Affective-News-Analysis-.
  • The authors claim this is the first transfer-learning (BERT) model for Arabic aspect polarity and report that it resists overfitting. Planned extensions replace the linear head with RNNs, self-attention or AEN.

Interesting ideas

  • I1. Encoding the query target as a second input segment lets one sentence receive a different label per aspect without any architectural change.
  • I2. Comparing a frozen encoder with a fine-tuned one is a cheap ablation showing how much the task-specific adaptation actually contributes.
  • I3. Start with a trivial head on a strong encoder before adding complex layers; the related work the paper cites found that sophisticated stacks on BERT did not reliably help.
  • I4. The case study contrasts negations that flip polarity ("not good for the elderly") with ones that do not ("not only suitable … but also close").
  • I5. Evaluation separates multi-aspect sentences with the same polarity from those with conflicting polarities, which exposes target-confusion errors.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): an attitude scorer in prag could take the clause plus a marked entity span, giving a per-participant polarity tied to a specific frame argument rather than a sentence-wide score.
  • I2 (learn): run gparse with frozen versus tuned encoder weights as a standing control, so gains are credited to the right component.
  • I3 (tree): establish a minimal linear-head baseline for gparse first, so any added parser machinery must prove its worth against it.
  • I4 (answer): build minimal-pair tests of negation scope as negative controls, checking that readers do not flip meaning on surface "not" alone.
  • I5 (world): test that the world model stores distinct evaluative states per entity when one sentence praises one entity and criticises another.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Sentiment & emotion Semitic languages

 Institutions

🎓Assiut UniversityAsyut · Egypt3 works · 234 citations

Cite

Cite as Mohammed M.Abdelgwad, Taysir Hassan A Soliman and Ahmed I.Taloba. 2021. Arabic aspect sentiment polarity classification using BERT. arXiv:2107.13290 · J Big Data 9, 115 (2022).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1186/s40537-022-00656-6 · arXiv 2107.13290

@misc{mabdelgwad2021arabic,
  title = {{Arabic aspect sentiment polarity classification using BERT}},
  author = {Mohammed M.Abdelgwad, Taysir Hassan A Soliman and Ahmed I.Taloba},
  year = {2021},
  doi = {10.1186/s40537-022-00656-6},
  eprint = {2107.13290},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1186/s40537-022-00656-6}
}

References

47 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Hu and B. Liu, “Mining and summarizing customer reviews,” in Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 168–177, 2004. find
  2. M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, “SemEval-2014 task 4: Aspect based sentiment analysis,” in Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), (Dublin, Ireland), pp. 27–35, Association for Computational Linguistics, Aug. 2014. our card
  3. M. Pontiki, D. Galanis, H. Papageorgiou, S. Manandhar, and I. Androutsopoulos, “Semeval-2015 task 12: Aspect based sentiment analysis,” in Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pp. 486–495, 2015. our card
  4. M. Pontiki, D. Galanis, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, M. Al-Smadi, M. Al-Ayyoub, Y. Zhao, B. Qin, O. De Clercq, et al., “Semeval-2016 task 5: Aspect based sentiment analysis,” in International workshop on semantic evaluation, pp. 19–30, 2016. our card
  5. K. A. Kwaik, M. Saad, S. Chatzikyriakidis, and S. Dobnik, “Lstm-cnn deep learning model for sentiment analysis of dialectal arabic,” in International Conference on Arabic Language Processing, pp. 108–121, Springer, 2019. find
  6. M. S. H. Ameur, F. Meziane, and A. Guessoum, “Arabic machine transliteration using an attention-based encoder-decoder model,” Procedia Computer Science, vol. 117, pp. 287–297, 2017. find
  7. M. Khalifa and K. Shaalan, “Character convolutions for arabic named entity recognition with long short-term memory networks,” Computer Speech & Language, vol. 58, pp. 335–346, 2019. find
  8. W. Algihab, N. Alawwad, A. Aldawish, and S. AlHumoud, “Arabic speech recognition with deep learning: A review,” in International Conference on Human-Computer Interaction, pp. 15–31, Springer, 2019. find
  9. E. Alsharhan and A. Ramsay, “Investigating the effects of gender, dialect, and training size on the performance of arabic speech recognition,” Language Resources and Evaluation, vol. 54, no. 4, pp. 975–998, 2020. find
  10. L. Jiang, M. Yu, M. Zhou, X. Liu, and T. Zhao, “Target-dependent Twitter sentiment classification,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, (Portland, Oregon, USA), pp. 151–160, Association for Computational Linguistics, June 2011. find
  11. S. Kiritchenko, X. Zhu, C. Cherry, and S. Mohammad, “Nrc-canada-2014: Detecting aspects and sentiment in customer reviews,” in Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp. 437–442, 2014. our card
  12. D. Tang, B. Qin, X. Feng, and T. Liu, “Effective lstms for target-dependent sentiment classification,” arXiv preprint arXiv:1512.01100, 2015. ↗ our card
  13. M. Zhang, Y. Zhang, and D.-T. Vo, “Gated neural networks for targeted sentiment analysis,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, 2016. find
  14. S. Poria, E. Cambria, and A. Gelbukh, “Aspect extraction for opinion mining with a deep convolutional neural network,” Knowledge-Based Systems, vol. 108, pp. 42–49, 2016. find
  15. Y. Wang, M. Huang, X. Zhu, and L. Zhao, “Attention-based lstm for aspect-level sentiment classification,” in Proceedings of the 2016 conference on empirical methods in natural language processing, pp. 606–615, 2016. our card
  16. M. Yang, W. Tu, J. Wang, F. Xu, and X. Chen, “Attention based lstm for target dependent sentiment classification,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, 2017. find
  17. P. Chen, Z. Sun, L. Bing, and W. Yang, “Recurrent attention network on memory for aspect sentiment analysis,” in Proceedings of the 2017 conference on empirical methods in natural language processing, pp. 452– 461, 2017. our card
  18. D. Ma, S. Li, X. Zhang, and H. Wang, “Interactive attention networks for aspect-level sentiment classification,” arXiv preprint arXiv:1709.00893, 2017. ↗
  19. J. Weston, S. Chopra, and A. Bordes, “Memory networks,” arXiv preprint arXiv:1410.3916, 2014. ↗
  20. D. Tang, B. Qin, and T. Liu, “Aspect level sentiment classification with deep memory network,” arXiv preprint arXiv:1605.08900, 2016. ↗ our card
  21. Y. Tay, L. A. Tuan, and S. C. Hui, “Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, 2018. find
  22. Y. Tay, L. A. Tuan, and S. C. Hui, “Dyadic memory networks for aspect-based sentiment analysis,” in Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 107–116, 2017. find
  23. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” arXiv preprint arXiv:1802.05365, 2018. ↗ our card
  24. A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” 2018. find
  25. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018. ↗ our card
  26. X. Li, L. Bing, W. Zhang, and W. Lam, “Exploiting bert for end-to-end aspect-based sentiment analysis,” arXiv preprint arXiv:1910.00883, 2019. ↗
  27. C. Sun, L. Huang, and X. Qiu, “Utilizing bert for aspect-based sentiment analysis via constructing auxiliary sentence,” arXiv preprint arXiv:1903.09588, 2019. ↗ our card
  28. Z. Gao, A. Feng, X. Song, and X. Wu, “Target-dependent sentiment classification with bert,” Ieee Access, vol. 7, pp. 154290–154299, 2019. find
  29. Y. Song, J. Wang, T. Jiang, Z. Liu, and Y. Rao, “Attentional encoder network for targeted sentiment classification,” arXiv preprint arXiv:1902.09314, 2019. ↗
  30. M. Hoang, O. A. Bihorac, and J. Rouces, “Aspect-based sentiment analysis using bert,” in Proceedings of the 22nd Nordic Conference on Computational Linguistics, pp. 187–196, 2019. find
  31. A. Bhoi and S. Joshi, “Various approaches to aspect-based sentiment analysis,” arXiv preprint arXiv:1805.01984, 2018. ↗
  32. H. Xu, B. Liu, L. Shu, and P. S. Yu, “Bert post-training for review reading comprehension and aspect-based sentiment analysis,” arXiv preprint arXiv:1904.02232, 2019. ↗ our card
  33. F. Nurifan, R. Sarno, and K. R. Sungkono, “Aspect based sentiment analysis for restaurant reviews using hybrid elmo-wikipedia and hybrid expanded opinion lexicon-senticircle,” International Journal of Intelligent Engineering and Systems, vol. 12, no. 6, pp. 47–58, 2019. find
  34. S. Ruder, P. Ghaffari, and J. G. Breslin, “Insight-1 at semeval-2016 task 5: Deep learning for multilingual aspect-based sentiment analysis,” arXiv preprint arXiv:1609.02748, 2016. ↗
  35. S. Ruder, P. Ghaffari, and J. G. Breslin, “A hierarchical model of reviews for aspect-based sentiment analysis,” arXiv preprint arXiv:1609.02745, 2016. ↗
  36. M. Al-Smadi, O. Qawasmeh, M. Al-Ayyoub, Y. Jararweh, and B. Gupta, “Deep recurrent neural network vs. support vector machine for aspect-based sentiment analysis of arabic hotels’ reviews,” Journal of computational science, vol. 27, pp. 386–393, 2018. find
  37. M. Al-Smadi, B. Talafha, M. Al-Ayyoub, and Y. Jararweh, “Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews,” International Journal of Machine Learning and Cybernetics, vol. 10, no. 8, pp. 2163–2175, 2019. find
  38. M. M. Abdelgwad, T. H. A. Soliman, A. I. Taloba, and M. F. Farghaly, “Arabic aspect based sentiment analysis using bidirectional gru based models,” arXiv preprint arXiv:2101.10539, 2021. ↗ our card
  39. S. Al-Dabet, S. Tedmori, and A.-S. Mohammad, “Enhancing arabic aspect-based sentiment analysis using deep learning models,” Computer Speech & Language, p. 101224, 2021. find
  40. A.-S. Mohammad, M. M. Hammad, A. Sa’ad, A.-T. Saja, and E. Cambria, “Gated recurrent unit with multilingual universal sentence encoder for arabic aspect-based sentiment analysis,” Knowledge-Based Systems, p. 107540, 2021. find
  41. R. Bensoltane and T. Zaki, “Towards arabic aspect-based sentiment analysis: a transfer learning-based approach,” Social Network Analysis and Mining, vol. 12, no. 1, pp. 1–16, 2022. find
  42. A. S. Fadel, M. E. Saleh, and O. A. Abulnaja, “Arabic aspect extraction based on stacked contextualized embedding with deep learning,” IEEE Access, vol. 10, pp. 30526–30535, 2022. find
  43. M. Al-Smadi, O. Qawasmeh, B. Talafha, and M. Quwaider, “Human annotated arabic dataset of book reviews for aspect based sentiment analysis,” in 2015 3rd International Conference on Future Internet of Things and Cloud, pp. 726–730, IEEE, 2015. find
  44. H. Al-Sarhan, M. Al-So’ud, M. Al-Smadi, M. Al-Ayyoub, and Y. Jararweh, “Framework for affective news analysis of arabic news: 2014 gaza attacks case study,” in 2016 7th International Conference on Information and Communication Systems (ICICS), pp. 327–332, IEEE, 2016. find
  45. A. Safaya, M. Abdullatif, and D. Yuret, “KUISAIL at SemEval-2020 task 12: BERT-CNN for offensive speech identification in social media,” in Proceedings of the Fourteenth Workshop on Semantic Evaluation, (Barcelona (online)), pp. 2054–2059, International Committee for Computational Linguistics, Dec. 2020. our card
  46. D. Tang, B. Qin, X. Feng, and T. Liu, “Effective lstms for target-dependent sentiment classification,” arXiv preprint arXiv:1512.01100, 2015. ↗ our card
  47. M. M.Abdelgwad, T. H. A Soliman, A. I.Taloba, and M. F. Farghaly, “Arabic aspect based sentiment analysis using bidirectional gru based models,” Journal of King Saud University - Computer and Information Sciences, 2021. our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/m-abdelgwad-2021-arabic-aspect-sentiment-polarity-classification/